Package ai.djl.serving.wlm.util

Class WlmConfigManager

  • public final class WlmConfigManager
extends java.lang.Object
    This manages some configurations used by the WorkLoadManager.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      int getBatchSize()
      Returns the default batchSize for workers.
      int getDefaultMaxWorkers​(ModelInfo<?,​?> modelInfo, ai.djl.Device device, int target)
      Returns the default maximum number of workers for a new registered model.
      int getDefaultMinWorkers​(ModelInfo<?,​?> modelInfo, ai.djl.Device device, int minWorkers, int maxWorkers)
      Returns the default minimum number of workers for a new registered model.
      static WlmConfigManager getInstance()
      Returns the singleton ConfigManager instance.
      int getJobQueueSize()
      Returns the default job queue size.
      int getMaxBatchDelay()
      Returns the default maxBatchDelay for the working queue.
      int getMaxIdleTime()
      Returns the default max idle time for workers.
      boolean isDebug()
      Returns if debug is enabled.
      void setBatchSize​(int batchSize)
      Sets the default batchSize for workers.
      void setJobQueueSize​(int jobQueueSize)
      Sets the default job queue size.
      void setMaxBatchDelay​(int maxBatchDelay)
      Sets the default maxBatchDelay for the working queue.
      void setMaxIdleTime​(int maxIdleTime)
      Sets the default max idle time for workers.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • WlmConfigManager

        public WlmConfigManager()

    • Method Detail

      • getInstance

        public static WlmConfigManager getInstance()
        Returns the singleton ConfigManager instance.
        Returns:
        the singleton ConfigManager instance

      • isDebug

        public boolean isDebug()
        Returns if debug is enabled.
        Returns:
        true if debug is enabled

      • getJobQueueSize

        public int getJobQueueSize()
        Returns the default job queue size.
        Returns:
        the default job queue size

      • setJobQueueSize

        public void setJobQueueSize​(int jobQueueSize)
        Sets the default job queue size.
        Parameters:
        jobQueueSize - the new default job queue size

      • getMaxIdleTime

        public int getMaxIdleTime()
        Returns the default max idle time for workers.
        Returns:
        the default max idle time

      • setMaxIdleTime

        public void setMaxIdleTime​(int maxIdleTime)
        Sets the default max idle time for workers.
        Parameters:
        maxIdleTime - the new default max idle time

      • getBatchSize

        public int getBatchSize()
        Returns the default batchSize for workers.
        Returns:
        the default max idle time

      • setBatchSize

        public void setBatchSize​(int batchSize)
        Sets the default batchSize for workers.
        Parameters:
        batchSize - the new default batchSize

      • getMaxBatchDelay

        public int getMaxBatchDelay()
        Returns the default maxBatchDelay for the working queue.
        Returns:
        the default max batch delay

      • setMaxBatchDelay

        public void setMaxBatchDelay​(int maxBatchDelay)
        Sets the default maxBatchDelay for the working queue.
        Parameters:
        maxBatchDelay - the new default maxBatchDelay

      • getDefaultMinWorkers

        public int getDefaultMinWorkers​(ModelInfo<?,​?> modelInfo,
                                ai.djl.Device device,
                                int minWorkers,
                                int maxWorkers)
        Returns the default minimum number of workers for a new registered model.
        Parameters:
        modelInfo - the ModelInfo
        device - the device that model loaded on
        minWorkers - the minimum number of workers of a new registered model
        maxWorkers - the maximum number of workers of a new registered model
        Returns:
        the calculated minimum number of workers for a new registered model

      • getDefaultMaxWorkers

        public int getDefaultMaxWorkers​(ModelInfo<?,​?> modelInfo,
                                ai.djl.Device device,
                                int target)
        Returns the default maximum number of workers for a new registered model.
        Parameters:
        modelInfo - the ModelInfo
        device - the device that model loaded on
        target - the target number of worker
        Returns:
        the default number of workers for a new registered model