Index

A B C D E F G H I J L M N O P Q R S T U V W
All Classes and Interfaces|All Packages|Serialized Form

A

Adapter - Class in ai.djl.serving.wlm

An adapter is a modification producing a variation of a model that can be used during prediction.

Adapter(String, String, Map<String, String>) - Constructor for class ai.djl.serving.wlm.Adapter

Constructs an Adapter.

addConfigJob(WorkerJob<I, O>) - Method in class ai.djl.serving.wlm.WorkerThread

Adds a configuration job to this thread.

addListener(ModelServerListener) - Method in class ai.djl.serving.wlm.util.EventManager

Adds the listener to the EventManager.

ai.djl.serving.wlm - package ai.djl.serving.wlm

Contains the model server backend which manages worker threads and executes jobs on models.

ai.djl.serving.wlm.util - package ai.djl.serving.wlm.util

Contains utilities to support the WorkLoadManager.

apply(List<I>) - Method in interface ai.djl.serving.wlm.JobFunction

Applies this function.

AutoIncIdGenerator - Class in ai.djl.serving.wlm.util

Class to generate an unique worker id.

AutoIncIdGenerator(String) - Constructor for class ai.djl.serving.wlm.util.AutoIncIdGenerator

Constructs an AutoIncIdGenerator.

B

batchSize - Variable in class ai.djl.serving.wlm.PermanentBatchAggregator

batchSize - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

build() - Method in class ai.djl.serving.wlm.WorkerThread.Builder

Builds the WorkerThread with the provided data.

builder(WorkerPoolConfig<I, O>) - Static method in class ai.djl.serving.wlm.WorkerThread

Creates a builder to build a WorkerThread.

C

cleanup() - Method in class ai.djl.serving.wlm.WorkerPool

removes all stopped workers and workers in state error from the pool.

close() - Method in class ai.djl.serving.wlm.ModelInfo

Close all loaded workers.

close() - Method in class ai.djl.serving.wlm.ModelInfo.ModelThread

Closes the thread type and frees any resources.

close() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Close all loaded workers.

close() - Method in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig

Closes the thread type and frees any resources.

close() - Method in class ai.djl.serving.wlm.WorkLoadManager

Close all wpcs related to the WorkloadManager.

configJobs - Variable in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig

configureWorkers(int, int) - Method in class ai.djl.serving.wlm.WorkerGroup

Configures minimum and maximum number of workers.

D

decreaseRef() - Method in class ai.djl.serving.wlm.WorkerPool

Decrease the reference count and return the current count.

drainTo(List<WorkerJob<I, O>>, long) - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

E

equals(Object) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

EventManager - Class in ai.djl.serving.wlm.util

A class manages model server events.

F

FAILED - Enum constant in enum class ai.djl.serving.wlm.WorkerPoolConfig.Status

G

generate() - Method in class ai.djl.serving.wlm.util.AutoIncIdGenerator

Generates a new worker id.

generateHuggingFaceConfigUri(ModelInfo<?, ?>, String) - Static method in class ai.djl.serving.wlm.LmiUtils

Returns the Huggingface config.json file URI.

generateNum() - Method in class ai.djl.serving.wlm.util.AutoIncIdGenerator

Generates a new worker id without the prefix.

getAdapter(String) - Method in class ai.djl.serving.wlm.ModelInfo

Returns an adapter on this ModelInfo.

getAdapters() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the adapters for this model.

getBatchSize() - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns the default batchSize for workers.

getBatchSize() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the configured batch size.

getConfigJobs() - Method in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig

Gets the configuration jobs for the worker.

getDevice() - Method in class ai.djl.serving.wlm.WorkerGroup

Returns the device of the worker group.

getDevice() - Method in class ai.djl.serving.wlm.WorkerThread

Returns the device used by the thread.

getEngine() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the engine.

getEngineName() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the engine name.

getFuture() - Method in class ai.djl.serving.wlm.util.WorkerJob

Returns the future for the job.

getId() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the worker configs ID.

getInput() - Method in class ai.djl.serving.wlm.Job

Returns the input data.

getInputClass() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the model input class.

getInstance() - Static method in class ai.djl.serving.wlm.util.EventManager

Returns a singleton EventManager instance.

getInstance() - Static method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns the singleton ConfigManager instance.

getJob() - Method in class ai.djl.serving.wlm.util.WorkerJob

Returns the Job.

getJobQueue() - Method in class ai.djl.serving.wlm.WorkerPool

Returns the JobQueue for this model.

getJobQueueSize() - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns the default job queue size.

getLoadOnDevices() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the devices the worker type will be loaded on at startup.

getLoadOnDevices() - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns the devices the model will be loaded on at startup.

getLoadOnDevices() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the devices the worker type will be loaded on at startup.

getMaxBatchDelayMillis() - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns the default max batch delay in milliseconds for the working queue.

getMaxBatchDelayMillis() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the maximum delay in milliseconds to aggregate a batch.

getMaxIdleSeconds() - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns the default max idle time for workers.

getMaxIdleSeconds() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the configured max idle time in seconds of workers.

getMaxWorkers() - Method in class ai.djl.serving.wlm.WorkerGroup

Returns the max number of workers for the model and device.

getMaxWorkers() - Method in class ai.djl.serving.wlm.WorkerPool

Returns the maximum number of workers for a model across all devices.

getMaxWorkers(Device) - Method in class ai.djl.serving.wlm.ModelInfo

Returns the maximum number of workers.

getMaxWorkers(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the maximum number of workers.

getMinWorkers() - Method in class ai.djl.serving.wlm.WorkerGroup

Returns the min number of workers for the model and device.

getMinWorkers(Device) - Method in class ai.djl.serving.wlm.ModelInfo

Returns the minimum number of workers.

getMinWorkers(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the minimum number of workers.

getModel(Device) - Method in class ai.djl.serving.wlm.ModelInfo

Returns the loaded ZooModel for a device.

getModels() - Method in class ai.djl.serving.wlm.ModelInfo

Returns all loaded models.

getModelUrl() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the worker type url.

getName() - Method in class ai.djl.serving.wlm.Adapter

Returns the adapter name.

getNumRunningWorkers(WorkerPoolConfig<?, ?>) - Method in class ai.djl.serving.wlm.WorkLoadManager

Returns the number of running workers of a wpc.

getOutput() - Method in class ai.djl.serving.wlm.Job

Returns the output data.

getOutputClass() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the model output class.

getPredictor() - Method in class ai.djl.serving.wlm.ModelInfo.ModelThread

Returns the predictor.

getProperties() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the properties of the model.

getQueueSize() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the configured size of the workers queue.

getRequest() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

Poll the queue and return a list of Input Objects for the model.

getReservedMemoryMb() - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns the default reserved memory in MB.

getRunner() - Method in class ai.djl.serving.wlm.Job

Returns the task to run for the job.

getSrc() - Method in class ai.djl.serving.wlm.Adapter

Returns the adapter src.

getStartTime() - Method in class ai.djl.serving.wlm.WorkerThread

Returns the thread start time.

getState() - Method in class ai.djl.serving.wlm.WorkerThread

Returns the worker state.

getStatus() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the worker type loading status.

getStatus() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the worker type loading status.

getThreadType() - Method in class ai.djl.serving.wlm.WorkerThread

Returns the WorkerPoolConfig's WorkerPoolConfig.ThreadConfig for this thread.

getUid() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the worker configs unique ID.

getVersion() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the worker type version.

getWaitingMicroSeconds() - Method in class ai.djl.serving.wlm.Job

Returns the wait time of this job.

getWorkerGroups() - Method in class ai.djl.serving.wlm.WorkerPool

Returns a map of WorkerGroup.

getWorkerId() - Method in class ai.djl.serving.wlm.WorkerThread

Returns the worker thread ID.

getWorkerIdNum() - Method in class ai.djl.serving.wlm.WorkerThread

Returns the worker thread ID (number without prefix).

getWorkerPool(WorkerPoolConfig<I, O>) - Method in class ai.djl.serving.wlm.WorkLoadManager

Returns the WorkerPool for a model.

getWorkerPoolById(String) - Method in class ai.djl.serving.wlm.WorkLoadManager

Returns the WorkerPool for a wpc.

getWorkers() - Method in class ai.djl.serving.wlm.WorkerGroup

Returns a list of workers.

getWorkers() - Method in class ai.djl.serving.wlm.WorkerPool

Returns a list of worker thread.

getWpc() - Method in class ai.djl.serving.wlm.Job

Returns the worker pool config that is associated with this job.

getWpc() - Method in class ai.djl.serving.wlm.WorkerPool

Returns the model of the worker pool.

H

hashCode() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

hasInputOutputClass(Class<I>, Class<O>) - Method in class ai.djl.serving.wlm.ModelInfo

Clarifies the input and output class when not specified.

I

id - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

increaseRef() - Method in class ai.djl.serving.wlm.WorkerPool

Increases the reference count.

inferModelNameFromUrl(String) - Static method in class ai.djl.serving.wlm.ModelInfo

Infer model name form model URL in case model name is not provided.

initialize() - Method in class ai.djl.serving.wlm.ModelInfo

Initialize the worker.

initialize() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Initialize the worker.

initWorkers(String) - Method in class ai.djl.serving.wlm.WorkerPool

Initializes new worker capacities for this model.

initWorkers(String, int, int) - Method in class ai.djl.serving.wlm.WorkerPool

Initializes new worker capacities for this model.

isAllWorkerBusy() - Method in class ai.djl.serving.wlm.WorkerPool

Returns true if all workers are busy.

isAllWorkerDied() - Method in class ai.djl.serving.wlm.WorkerPool

Return if all workers died.

isDebug() - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Returns if debug is enabled.

isFinished() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

Checks if this BatchAggregator and the thread can be shutdown or if this aggregator waits for more data.

isFinished() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator

Checks if this BatchAggregator and the thread can be shutdown or if this aggregator waits for more data.

isFixPoolThread() - Method in class ai.djl.serving.wlm.WorkerThread

check if this worker is instantiate is one of the fix threads of a pool.

isFullyScaled() - Method in class ai.djl.serving.wlm.WorkerPool

Returns if the worker groups is fully scaled.

isParallelLoading() - Method in class ai.djl.serving.wlm.ModelInfo

Returns if the worker type can be load parallel on multiple devices.

isParallelLoading() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns if the worker type can be load parallel on multiple devices.

isRunning() - Method in class ai.djl.serving.wlm.WorkerThread

Returns true if the worker thread is running.

J

Job<I,O> - Class in ai.djl.serving.wlm

A class represents an inference job.

Job(WorkerPoolConfig<I, O>, I) - Constructor for class ai.djl.serving.wlm.Job

Constructs a new Job instance.

Job(WorkerPoolConfig<I, O>, I, JobFunction<I, O>) - Constructor for class ai.djl.serving.wlm.Job

Constructs a new Job instance.

JobFunction<I,O> - Interface in ai.djl.serving.wlm

A function describing the action to take in a Job.

jobQueue - Variable in class ai.djl.serving.wlm.PermanentBatchAggregator

L

LmiConfigRecommender - Class in ai.djl.serving.wlm

A utility class to auto configure LMI model properties.

LmiUtils - Class in ai.djl.serving.wlm

A utility class to detect optimal engine for LMI model.

load(Device) - Method in class ai.djl.serving.wlm.ModelInfo

Loads the worker type to the specified device.

load(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Loads the worker type to the specified device.

M

maxBatchDelayMicros - Variable in class ai.djl.serving.wlm.PermanentBatchAggregator

maxBatchDelayMillis - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

maxIdleSeconds - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

maxWorkers - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

minWorkers - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

ModelInfo<I,O> - Class in ai.djl.serving.wlm

A class represent a loaded model and it's metadata.

ModelInfo(String) - Constructor for class ai.djl.serving.wlm.ModelInfo

Constructs a new ModelInfo instance.

ModelInfo(String, String, Criteria<I, O>) - Constructor for class ai.djl.serving.wlm.ModelInfo

Constructs a ModelInfo based on a Criteria.

ModelInfo(String, String, String, String, String, Class<I>, Class<O>, int, int, int, int, int, int) - Constructor for class ai.djl.serving.wlm.ModelInfo

Constructs a new ModelInfo instance.

ModelInfo.ModelThread - Class in ai.djl.serving.wlm

ModelServerListener - Interface in ai.djl.serving.wlm.util

An interface that represent a model server event listener.

ModelServerListenerAdapter - Class in ai.djl.serving.wlm.util

Base implementation of the ModelServerListener that does nothing.

ModelServerListenerAdapter() - Constructor for class ai.djl.serving.wlm.util.ModelServerListenerAdapter

ModelThread(Device) - Constructor for class ai.djl.serving.wlm.ModelInfo.ModelThread

modelUrl - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

N

name - Variable in class ai.djl.serving.wlm.Adapter

newInstance(WorkerPoolConfig<?, ?>, String, String, Map<String, String>) - Static method in class ai.djl.serving.wlm.Adapter

Constructs a new Adapter.

newThread(Device) - Method in class ai.djl.serving.wlm.ModelInfo

Starts a new WorkerThread for this WorkerPoolConfig.

newThread(Device) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Starts a new WorkerThread for this WorkerPoolConfig.

O

onAdapterLoaded(ModelInfo<?, ?>, Adapter) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when adapter loading finished.

onAdapterLoaded(ModelInfo<?, ?>, Adapter) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when adapter loading finished.

onAdapterLoaded(ModelInfo<?, ?>, Adapter) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when adapter loading finished.

onAdapterLoading(ModelInfo<?, ?>, Path) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when adapter loading start.

onAdapterLoading(ModelInfo<?, ?>, Path) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when adapter loading start.

onAdapterLoading(ModelInfo<?, ?>, Path) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when adapter loading start.

onModelConfigured(ModelInfo<?, ?>) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when model properties configuration finished.

onModelConfigured(ModelInfo<?, ?>) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when model properties configuration finished.

onModelConfigured(ModelInfo<?, ?>) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when model properties configuration finished.

onModelConverted(ModelInfo<?, ?>, String) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when model conversion finished.

onModelConverted(ModelInfo<?, ?>, String) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when model conversion finished.

onModelConverted(ModelInfo<?, ?>, String) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when model conversion finished.

onModelConverting(ModelInfo<?, ?>, String) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when model conversion started.

onModelConverting(ModelInfo<?, ?>, String) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when model conversion started.

onModelConverting(ModelInfo<?, ?>, String) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when model conversion started.

onModelDownloaded(ModelInfo<?, ?>, Path) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when model downloading finished.

onModelDownloaded(ModelInfo<?, ?>, Path) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when model downloading finished.

onModelDownloaded(ModelInfo<?, ?>, Path) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when model downloading finished.

onModelDownloading(ModelInfo<?, ?>) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when model downloading started.

onModelDownloading(ModelInfo<?, ?>) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when model downloading started.

onModelDownloading(ModelInfo<?, ?>) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when model downloading started.

onModelLoaded(ModelInfo<?, ?>) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when model loading finished.

onModelLoaded(ModelInfo<?, ?>) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when model loading finished.

onModelLoaded(ModelInfo<?, ?>) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when model loading finished.

onModelLoading(ModelInfo<?, ?>, Device) - Method in class ai.djl.serving.wlm.util.EventManager

Invoked when model loading start.

onModelLoading(ModelInfo<?, ?>, Device) - Method in interface ai.djl.serving.wlm.util.ModelServerListener

Invoked when model loading start.

onModelLoading(ModelInfo<?, ?>, Device) - Method in class ai.djl.serving.wlm.util.ModelServerListenerAdapter

Invoked when model loading start.

optFixPoolThread(boolean) - Method in class ai.djl.serving.wlm.WorkerThread.Builder

Sets if the workerThread should be part of the fixed pool.

options - Variable in class ai.djl.serving.wlm.Adapter

P

PENDING - Enum constant in enum class ai.djl.serving.wlm.WorkerPoolConfig.Status

PermanentBatchAggregator<I,O> - Class in ai.djl.serving.wlm

a batch aggregator that never terminates by itself.

PermanentBatchAggregator(WorkerPoolConfig<I, O>, LinkedBlockingDeque<WorkerJob<I, O>>) - Constructor for class ai.djl.serving.wlm.PermanentBatchAggregator

Constructs a PermanentBatchAggregator instance.

pollBatch() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

Fills in the list with a batch of jobs.

pollBatch() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator

Fills in the list with a batch of jobs.

PyAdapter - Class in ai.djl.serving.wlm

An overload of Adapter for the python engine.

PyAdapter(String, String, Map<String, String>) - Constructor for class ai.djl.serving.wlm.PyAdapter

Constructs an Adapter.

Q

queueSize - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

R

READY - Enum constant in enum class ai.djl.serving.wlm.WorkerPoolConfig.Status

register(WorkerPool<I, O>) - Method in class ai.djl.serving.wlm.Adapter

Registers this adapter in a worker pool.

registerAdapter(Adapter) - Method in class ai.djl.serving.wlm.ModelInfo

Adds an adapter to this ModelInfo.

registerJob(WorkerPoolConfig<I, O>, WorkerPoolConfig.ThreadConfig<I, O>) - Method in class ai.djl.serving.wlm.Adapter

Creates a WorkerJob to register this adapter in a WorkerThread.

registerPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.Adapter

registerPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.PyAdapter

registerWorkerPool(WorkerPoolConfig<I, O>) - Method in class ai.djl.serving.wlm.WorkLoadManager

Registers a WorkerPool (model).

run() - Method in class ai.djl.serving.wlm.WorkerThread

run(List<Job<I, O>>) - Method in class ai.djl.serving.wlm.ModelInfo.ModelThread

run(List<Job<I, O>>) - Method in class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig

Runs the work on the WorkerThread and stores in the job.

runAll(List<Job<I, O>>, JobFunction<I, O>) - Static method in class ai.djl.serving.wlm.Job

Runs a JobFunction on a batch of jobs and sets the result in their output.

runJob(Job<I, O>) - Method in class ai.djl.serving.wlm.WorkLoadManager

Adds an inference job to the job queue of the next free worker.

S

SageMakerUtils - Class in ai.djl.serving.wlm

A utility class to detect optimal engine for SageMaker saved model.

scaleWorkers(String, int, int) - Method in class ai.djl.serving.wlm.WorkerPool

Sets new worker capacities for this model.

sendError(Throwable) - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

Completes the job with an error.

sendResponse() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

Sends to response to all waiting clients.

setBatchSize(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Sets the default batchSize for workers.

setBatchSize(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the configured batch size.

setDevice(Device) - Method in class ai.djl.serving.wlm.WorkerThread.Builder

RSets the device to run operations on.

setFailOutput(Job<Input, Output>, int, String) - Static method in class ai.djl.serving.wlm.Job

Sets a Job output to a failure.

setId(String) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the worker configs ID.

setJobQueue(LinkedBlockingDeque<WorkerJob<I, O>>) - Method in class ai.djl.serving.wlm.WorkerThread.Builder

Sets the jobQueue used to poll for new jobs.

setJobQueueSize(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Sets the default job queue size.

setLoadOnDevices(String) - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Sets the devices the model will be loaded on at startup.

setMaxBatchDelayMillis(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Sets the default max batch delay in milliseconds for the working queue.

setMaxBatchDelayMillis(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the maximum delay in milliseconds to aggregate a batch.

setMaxIdleSeconds(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Sets the default max idle time in seconds for workers.

setMaxIdleSeconds(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the configured max idle time in seconds of workers.

setMaxWorkers(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the starting number of max workers.

setMinMaxWorkers(int, int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the starting minimum and maximum number of workers.

setMinWorkers(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the starting number of min workers.

setOutput(O) - Method in class ai.djl.serving.wlm.Job

Sets the output of the job.

setQueueSize(int) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Sets the configured size of the workers queue.

setReservedMemoryMb(int) - Method in class ai.djl.serving.wlm.util.WlmConfigManager

Sets the reserved memory in MB.

shutdown() - Method in class ai.djl.serving.wlm.WorkerPool

Shuts down all the worker threads in the work pool.

shutdown(WorkerState) - Method in class ai.djl.serving.wlm.WorkerThread

Shuts down the worker thread.

shutdownWorkers() - Method in class ai.djl.serving.wlm.WorkerPool

Shutdown all works.

src - Variable in class ai.djl.serving.wlm.Adapter

stripPrefix(String) - Method in class ai.djl.serving.wlm.util.AutoIncIdGenerator

Removes the prefix to a generated Id.

T

TemporaryBatchAggregator<I,O> - Class in ai.djl.serving.wlm

a batch aggregator that terminates after a maximum idle time.

TemporaryBatchAggregator(WorkerPoolConfig<I, O>, LinkedBlockingDeque<WorkerJob<I, O>>) - Constructor for class ai.djl.serving.wlm.TemporaryBatchAggregator

a batch aggregator that terminates after a maximum idle time.

ThreadConfig(Device) - Constructor for class ai.djl.serving.wlm.WorkerPoolConfig.ThreadConfig

toString() - Method in class ai.djl.serving.wlm.WorkerPoolConfig

U

uid - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

unregister(WorkerPool<I, O>, String) - Static method in class ai.djl.serving.wlm.Adapter

Unregisters an adapter in a worker pool.

unregisterAdapter(String) - Method in class ai.djl.serving.wlm.ModelInfo

Removes an adapter from this ModelInfo.

unregisterJob(WorkerPoolConfig<I, O>, WorkerPoolConfig.ThreadConfig<I, O>) - Method in class ai.djl.serving.wlm.Adapter

Creates a WorkerJob to unregister this adapter from a WorkerThread.

unregisterPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.Adapter

unregisterPredictor(Predictor<?, ?>) - Method in class ai.djl.serving.wlm.PyAdapter

unregisterWorkerPool(WorkerPoolConfig<?, ?>) - Method in class ai.djl.serving.wlm.WorkLoadManager

Removes a worker pool from management.

V

valueOf(String) - Static method in enum class ai.djl.serving.wlm.WorkerPoolConfig.Status

Returns the enum constant of this class with the specified name.

valueOf(String) - Static method in enum class ai.djl.serving.wlm.WorkerState

Returns the enum constant of this class with the specified name.

values() - Static method in enum class ai.djl.serving.wlm.WorkerPoolConfig.Status

Returns an array containing the constants of this enum class, in the order they are declared.

values() - Static method in enum class ai.djl.serving.wlm.WorkerState

Returns an array containing the constants of this enum class, in the order they are declared.

version - Variable in class ai.djl.serving.wlm.WorkerPoolConfig

W

withDefaultDevice(String) - Method in class ai.djl.serving.wlm.ModelInfo

Returns the default device for this model if device is null.

withDefaultDevice(String) - Method in class ai.djl.serving.wlm.WorkerPoolConfig

Returns the default device for this model if device is null.

wjs - Variable in class ai.djl.serving.wlm.PermanentBatchAggregator

WlmCapacityException - Exception in ai.djl.serving.wlm.util

Thrown to throttle when a job is run but the job queue capacity is exceeded.

WlmCapacityException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmCapacityException

Constructs a WlmCapacityException with the specified detail message.

WlmCapacityException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmCapacityException

Constructs a WlmCapacityException with the specified detail message and cause.

WlmConfigManager - Class in ai.djl.serving.wlm.util

This manages some configurations used by the WorkLoadManager.

WlmException - Exception in ai.djl.serving.wlm.util

Thrown when an exception occurs inside the WorkLoadManager.

WlmException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmException

Constructs a WlmException with the specified detail message.

WlmException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmException

Constructs a WlmException with the specified detail message and cause.

WlmOutOfMemoryException - Exception in ai.djl.serving.wlm.util

Thrown when no enough memory to load the model.

WlmOutOfMemoryException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmOutOfMemoryException

Constructs a WlmOutOfMemoryException with the specified detail message.

WlmShutdownException - Exception in ai.djl.serving.wlm.util

Thrown when a job is run but all workers are shutdown.

WlmShutdownException(String) - Constructor for exception ai.djl.serving.wlm.util.WlmShutdownException

Constructs a WlmShutdownException with the specified detail message.

WlmShutdownException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.util.WlmShutdownException

Constructs a WlmShutdownException with the specified detail message and cause.

WORKER_BUSY - Enum constant in enum class ai.djl.serving.wlm.WorkerState

WORKER_ERROR - Enum constant in enum class ai.djl.serving.wlm.WorkerState

WORKER_MODEL_LOADED - Enum constant in enum class ai.djl.serving.wlm.WorkerState

WORKER_SCALED_DOWN - Enum constant in enum class ai.djl.serving.wlm.WorkerState

WORKER_STARTED - Enum constant in enum class ai.djl.serving.wlm.WorkerState

WORKER_STOPPED - Enum constant in enum class ai.djl.serving.wlm.WorkerState

WorkerGroup<I,O> - Class in ai.djl.serving.wlm

The WorkerGroup manages the WorkerPool for a particular Device.

WorkerJob<I,O> - Class in ai.djl.serving.wlm.util

A Job containing metadata from the WorkLoadManager.

WorkerJob(Job<I, O>, CompletableFuture<O>) - Constructor for class ai.djl.serving.wlm.util.WorkerJob

Constructs a new WorkerJob.

WorkerPool<I,O> - Class in ai.djl.serving.wlm

Manages the work load for a single model.

WorkerPoolConfig<I,O> - Class in ai.djl.serving.wlm

A WorkerPoolConfig represents a task that could be run in the WorkLoadManager.

WorkerPoolConfig() - Constructor for class ai.djl.serving.wlm.WorkerPoolConfig

WorkerPoolConfig.Status - Enum Class in ai.djl.serving.wlm

An enum represents state of a worker type.

WorkerPoolConfig.ThreadConfig<I,O> - Class in ai.djl.serving.wlm

The part of the WorkerPoolConfig for an individual WorkerThread.

WorkerState - Enum Class in ai.djl.serving.wlm

An enum represents state of a worker.

WorkerThread<I,O> - Class in ai.djl.serving.wlm

The WorkerThread is the worker managed by the WorkLoadManager.

WorkerThread.Builder<I,O> - Class in ai.djl.serving.wlm

A Builder to construct a WorkerThread.

WorkLoadManager - Class in ai.djl.serving.wlm

WorkLoadManager is responsible to manage the work load of worker thread.

WorkLoadManager() - Constructor for class ai.djl.serving.wlm.WorkLoadManager

Constructs a WorkLoadManager instance.

A B C D E F G H I J L M N O P Q R S T U V W
All Classes and Interfaces|All Packages|Serialized Form