Package ai.djl.serving.wlm

Class WorkLoadManager.WorkerPool

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    Enclosing class:
    WorkLoadManager
    public final class WorkLoadManager.WorkerPool
extends java.lang.Object
implements java.lang.AutoCloseable
    Manages the work load for a single model.

    • Constructor Summary

      Constructors 
      Constructor Description
      WorkerPool​(ModelInfo model)
      Construct and initial data structure.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void cleanup()
      removes all stopped workers and workers in state error from the pool.
      void close()
      java.util.concurrent.LinkedBlockingDeque<WorkerJob> getJobQueue()
      Returns the JobQueue for this model.
      int getMaxWorkers()
      Returns the maximum number of workers for a model.
      int getMinWorkers()
      Returns the minimum number of workers for a model.
      java.util.List<WorkerThread> getWorkers()
      Returns a list of worker thread.
      void log()
      Logs the current state of this WorkerPool when level "Debug" is enabled.
      WorkLoadManager.WorkerPool scaleWorkers​(java.lang.String deviceName, int newMinWorkers, int newMaxWorkers)
      Sets new worker capcities for this model.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • WorkerPool

        public WorkerPool​(ModelInfo model)
        Construct and initial data structure.
        Parameters:
        model - the model this WorkerPool belongs to.

    • Method Detail

      • getWorkers

        public java.util.List<WorkerThread> getWorkers()
        Returns a list of worker thread.
        Returns:
        the workers

      • getJobQueue

        public java.util.concurrent.LinkedBlockingDeque<WorkerJob> getJobQueue()
        Returns the JobQueue for this model.
        Returns:
        the jobQueue

      • getMinWorkers

        public int getMinWorkers()
        Returns the minimum number of workers for a model.
        Returns:
        the minimum number of workers for a model

      • getMaxWorkers

        public int getMaxWorkers()
        Returns the maximum number of workers for a model.
        Returns:
        the maximum number of workers for a model

      • scaleWorkers

        public WorkLoadManager.WorkerPool scaleWorkers​(java.lang.String deviceName,
                                               int newMinWorkers,
                                               int newMaxWorkers)
        Sets new worker capcities for this model.
        Parameters:
        deviceName - the device for the model
        newMinWorkers - minimum amount of workers.
        newMaxWorkers - maximum amount of workers.
        Returns:
        this ModelInfo

      • log

        public void log()
        Logs the current state of this WorkerPool when level "Debug" is enabled.

        Logs all thread-ids in the pool.

      • cleanup

        public void cleanup()
        removes all stopped workers and workers in state error from the pool.

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable