Package ai.djl.serving.wlm

Class WorkerThread.Builder<I,O>

java.lang.Object

ai.djl.serving.wlm.WorkerThread.Builder<I,O>

Enclosing class:

WorkerThread<I,O>

public static class WorkerThread.Builder<I,O>
extends java.lang.Object

A Builder to construct a WorkerThread.

Method Summary

Modifier and Type	Method	Description
WorkerThread<I,O>	build()	Builds the WorkerThread with the provided data.
WorkerThread.Builder<I,O>	optAggregator(ai.djl.serving.wlm.BatchAggregator<I,O> aggregator)	Sets a BatchAggregator which overrides the instantiated default BatchAggregator.
WorkerThread.Builder<I,O>	optFixPoolThread(boolean fixPoolThread)	Sets if the workerThread should be part of the fixed pool.
protected void	preBuildProcessing()
protected WorkerThread.Builder<I,O>	self()	Returns self reference to this builder.
WorkerThread.Builder<I,O>	setDevice(ai.djl.Device device)	RSets the device to run operations on.
WorkerThread.Builder<I,O>	setJobQueue(java.util.concurrent.LinkedBlockingDeque<WorkerJob<I,O>> jobQueue)	Sets the jobQueue used to poll for new jobs.
WorkerThread.Builder<I,O>	setModel(ModelInfo<I,O> model)	Sets the ModelInfo the thread will be responsible for.
protected void	validate()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

self

protected WorkerThread.Builder<I,O> self()

Returns self reference to this builder.

Returns:

self reference to this builder

preBuildProcessing

protected void preBuildProcessing()

validate

protected void validate()

build

public WorkerThread<I,O> build()

Builds the WorkerThread with the provided data.

Returns:

an WorkerThread

setModel

public WorkerThread.Builder<I,O> setModel(ModelInfo<I,O> model)

Sets the ModelInfo the thread will be responsible for.

Parameters:

model - the model to set

Returns:

self-reference to this builder.

setDevice

public WorkerThread.Builder<I,O> setDevice(ai.djl.Device device)

RSets the device to run operations on.

Parameters:

device - the device to run operations on

Returns:

self-reference to this builder

optAggregator

public WorkerThread.Builder<I,O> optAggregator(ai.djl.serving.wlm.BatchAggregator<I,O> aggregator)

Sets a BatchAggregator which overrides the instantiated default BatchAggregator.

Parameters:

aggregator - the BatchAggregator to set

Returns:

self-reference to this builder.

setJobQueue

public WorkerThread.Builder<I,O> setJobQueue(java.util.concurrent.LinkedBlockingDeque<WorkerJob<I,O>> jobQueue)

Sets the jobQueue used to poll for new jobs. The jobQueue is passed to the created standard BatchAggregators if the Batch-Aggregator is not override using optAggregator(BatchAggregator)

Parameters:

jobQueue - the jobQueue to set

Returns:

self-reference to this builder.

optFixPoolThread

public WorkerThread.Builder<I,O> optFixPoolThread(boolean fixPoolThread)

Sets if the workerThread should be part of the fixed pool. Fixed Pool workers don't terminate themself but are managed by WorkLoadManager min/max-worker scale functionality.

Parameters:

fixPoolThread - the fixPoolThread to set

Returns:

self-reference to this builder.