Package ai.djl.serving.wlm.util

Class WlmConfigManager

  • public final class WlmConfigManager
extends java.lang.Object
    This manages some configurations used by the WorkLoadManager.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      int getBatchSize()
      Returns the default batchSize for workers.
      int getDefaultMaxWorkers​(ai.djl.Model model)
      Returns the default maximum number of workers for a new registered model.
      int getDefaultMinWorkers​(ai.djl.Model model)
      Returns the default minimum number of workers for a new registered model.
      static WlmConfigManager getInstance()
      Returns the singleton ConfigManager instance.
      int getJobQueueSize()
      Returns the default job queue size.
      java.lang.String getLoadOnDevices()
      Returns the devices the model will be loaded on at startup.
      int getMaxBatchDelayMillis()
      Returns the default max batch delay in milliseconds for the working queue.
      int getMaxIdleSeconds()
      Returns the default max idle time for workers.
      int getReservedMemoryMb()
      Returns the default reserved memory in MB.
      boolean isDebug()
      Returns if debug is enabled.
      void setBatchSize​(int batchSize)
      Sets the default batchSize for workers.
      void setJobQueueSize​(int jobQueueSize)
      Sets the default job queue size.
      void setLoadOnDevices​(java.lang.String loadOnDevices)
      Sets the devices the model will be loaded on at startup.
      void setMaxBatchDelayMillis​(int maxBatchDelayMillis)
      Sets the default max batch delay in milliseconds for the working queue.
      void setMaxIdleSeconds​(int maxIdleSeconds)
      Sets the default max idle time in seconds for workers.
      void setReservedMemoryMb​(int reservedMemoryMb)
      Sets the reserved memory in MB.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • getInstance

        public static WlmConfigManager getInstance()
        Returns the singleton ConfigManager instance.
        Returns:
        the singleton ConfigManager instance

      • isDebug

        public boolean isDebug()
        Returns if debug is enabled.
        Returns:
        true if debug is enabled

      • getJobQueueSize

        public int getJobQueueSize()
        Returns the default job queue size.
        Returns:
        the default job queue size

      • setJobQueueSize

        public void setJobQueueSize​(int jobQueueSize)
        Sets the default job queue size.
        Parameters:
        jobQueueSize - the new default job queue size

      • getMaxIdleSeconds

        public int getMaxIdleSeconds()
        Returns the default max idle time for workers.
        Returns:
        the default max idle time

      • setMaxIdleSeconds

        public void setMaxIdleSeconds​(int maxIdleSeconds)
        Sets the default max idle time in seconds for workers.
        Parameters:
        maxIdleSeconds - the new default max idle time in seconds

      • getBatchSize

        public int getBatchSize()
        Returns the default batchSize for workers.
        Returns:
        the default max idle time

      • setBatchSize

        public void setBatchSize​(int batchSize)
        Sets the default batchSize for workers.
        Parameters:
        batchSize - the new default batchSize

      • getMaxBatchDelayMillis

        public int getMaxBatchDelayMillis()
        Returns the default max batch delay in milliseconds for the working queue.
        Returns:
        the default max batch delay in milliseconds

      • setMaxBatchDelayMillis

        public void setMaxBatchDelayMillis​(int maxBatchDelayMillis)
        Sets the default max batch delay in milliseconds for the working queue.
        Parameters:
        maxBatchDelayMillis - the new default max batch delay in milliseconds

      • getReservedMemoryMb

        public int getReservedMemoryMb()
        Returns the default reserved memory in MB.
        Returns:
        the default reserved memory in MB

      • setReservedMemoryMb

        public void setReservedMemoryMb​(int reservedMemoryMb)
        Sets the reserved memory in MB.
        Parameters:
        reservedMemoryMb - the reserved memory in MB

      • getLoadOnDevices

        public java.lang.String getLoadOnDevices()
        Returns the devices the model will be loaded on at startup.
        Returns:
        the devices the model will be loaded on at startup

      • setLoadOnDevices

        public void setLoadOnDevices​(java.lang.String loadOnDevices)
        Sets the devices the model will be loaded on at startup.
        Parameters:
        loadOnDevices - thes the default model will be loaded on at startup

      • getDefaultMinWorkers

        public int getDefaultMinWorkers​(ai.djl.Model model)
        Returns the default minimum number of workers for a new registered model.
        Parameters:
        model - the loaded Model
        Returns:
        the calculated minimum number of workers for a new registered model

      • getDefaultMaxWorkers

        public int getDefaultMaxWorkers​(ai.djl.Model model)
        Returns the default maximum number of workers for a new registered model.
        Parameters:
        model - the loaded Model
        Returns:
        the default number of workers for a new registered model