All Classes

Class	Description
Job<I,O>	A class represents an inference job.
ModelInfo<I,O>	A class represent a loaded model and it's metadata.
ModelInfo.Status	An enum represents state of a model.
PermanentBatchAggregator<I,O>	a batch aggregator that never terminates by itself.
TemporaryBatchAggregator<I,O>	a batch aggregator that terminates after a maximum idle time.
WlmCapacityException	Thrown to throttle when a job is run but the job queue capacity is exceeded.
WlmConfigManager	This manages some configurations used by the WorkLoadManager.
WlmException	Thrown when an exception occurs inside the WorkLoadManager.
WlmShutdownException	Thrown when a job is run but all workers are shutdown.
WorkerGroup<I,O>	The WorkerGroup manages the WorkerPool for a particular Device.
WorkerIdGenerator	class to generate an unique worker id.
WorkerJob<I,O>	A Job containing metadata from the WorkLoadManager.
WorkerPool<I,O>	Manages the work load for a single model.
WorkerState	An enum represents state of a worker.
WorkerThread<I,O>	The WorkerThread is the worker managed by the WorkLoadManager.
WorkerThread.Builder<I,O>	A Builder to construct a WorkerThread.
WorkLoadManager	WorkLoadManager is responsible to manage the work load of worker thread.