Package ai.djl.serving.wlm

Contains the model server backend which manages worker threads and executes jobs on models.

See: Description

Class Summary

Class	Description
Job	A class represents an inference job.
ModelInfo	A class represent a loaded model and it's metadata.
PermanentBatchAggregator	a batch aggregator that never terminates by itself.
TemporaryBatchAggregator	a batch aggregator that terminates after a maximum idle time.
WorkerIdGenerator	class to generate an unique worker id.
WorkerThread	The WorkerThread is the worker managed by the WorkLoadManager.
WorkerThread.Builder	A Builder to construct a WorkerThread.
WorkLoadManager	WorkLoadManager is responsible to manage the work load of worker thread.

Enum Summary

Enum	Description
WorkerState	An enum represents state of a worker.

Package ai.djl.serving.wlm Description

Contains the model server backend which manages worker threads and executes jobs on models.

See Also:

WorkLoadManager