ai.djl.serving.wlm

Class WorkLoadManager.WorkerPool

java.lang.Object

ai.djl.serving.wlm.WorkLoadManager.WorkerPool

Enclosing class:

WorkLoadManager

public final class WorkLoadManager.WorkerPool
extends java.lang.Object

Manages the work load for a single model.

Constructor Summary

Constructors

Constructor and Description
WorkerPool(ModelInfo model) Construct and initial data structure.

Method Summary

Modifier and Type	Method and Description
void	cleanup() removes all stopped workers and workers in state error from the pool.
java.util.concurrent.LinkedBlockingDeque<WorkerJob>	getJobQueue() Returns the JobQueue for this model.
int	getMaxWorkers() Returns the maximum number of workers for a model.
int	getMinWorkers() Returns the minimum number of workers for a model.
java.util.List<WorkerThread>	getWorkers() Returns a list of worker thread.
void	log() Logs the current state of this WorkerPool when level "Debug" is enabled.
WorkLoadManager.WorkerPool	scaleWorkers(java.lang.String deviceName, int newMinWorkers, int newMaxWorkers) Sets new worker capcities for this model.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WorkerPool

public WorkerPool(ModelInfo model)

Construct and initial data structure.

Parameters:

model - the model this WorkerPool belongs to.

Method Detail

getWorkers

public java.util.List<WorkerThread> getWorkers()

Returns a list of worker thread.

Returns:

the workers

getJobQueue

public java.util.concurrent.LinkedBlockingDeque<WorkerJob> getJobQueue()

Returns the JobQueue for this model.

Returns:

the jobQueue

getMinWorkers

public int getMinWorkers()

Returns the minimum number of workers for a model.

Returns:

the minimum number of workers for a model

getMaxWorkers

public int getMaxWorkers()

Returns the maximum number of workers for a model.

Returns:

the maximum number of workers for a model

scaleWorkers

public WorkLoadManager.WorkerPool scaleWorkers(java.lang.String deviceName,
                                               int newMinWorkers,
                                               int newMaxWorkers)

Sets new worker capcities for this model.

Parameters:

deviceName - the device for the model

newMinWorkers - minimum amount of workers.

newMaxWorkers - maximum amount of workers.

Returns:

this ModelInfo

log

public void log()

Logs the current state of this WorkerPool when level "Debug" is enabled.

Logs all thread-ids in the pool.

cleanup

public void cleanup()

removes all stopped workers and workers in state error from the pool.