Package ai.djl.serving.wlm

Class ModelInfo<I,​O>

  • public final class ModelInfo<I,​O>
extends java.lang.Object
    A class represent a loaded model and it's metadata.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  ModelInfo.Status
      An enum represents state of a model.

    • Constructor Summary

      Constructors 
      Constructor Description
      ModelInfo​(java.lang.String id, ai.djl.repository.zoo.Criteria<I,​O> criteria)
      Constructs a ModelInfo based on a Criteria.
      ModelInfo​(java.lang.String modelUrl, java.lang.Class<I> inputClass, java.lang.Class<O> outputClass)
      Constructs a new ModelInfo instance.
      ModelInfo​(java.lang.String id, java.lang.String modelUrl, java.lang.String version, java.lang.String engineName, java.lang.Class<I> inputClass, java.lang.Class<O> outputClass, int queueSize, int maxIdleSeconds, int maxBatchDelayMillis, int batchSize)
      Constructs a new ModelInfo instance.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Close all loaded models.
      static ai.djl.util.Pair<java.lang.String,​java.nio.file.Path> downloadModel​(java.lang.String modelUrl)
      Downloads model from the model URL.
      boolean equals​(java.lang.Object o)
      int getBatchSize()
      Returns the configured batch size.
      java.lang.String getEngineName()
      Returns the engine name.
      java.lang.Class<I> getInputClass()
      Returns the model input class.
      int getMaxBatchDelayMillis()
      Returns the maximum delay in milliseconds to aggregate a batch.
      int getMaxIdleSeconds()
      Returns the configured max idle time in seconds of workers.
      ai.djl.repository.zoo.ZooModel<I,​O> getModel​(ai.djl.Device device)
      Returns the loaded ZooModel for a device.
      java.lang.String getModelId()
      Returns the model ID.
      java.util.Map<ai.djl.Device,​ai.djl.repository.zoo.ZooModel<I,​O>> getModels()
      Returns all loaded models.
      java.lang.String getModelUrl()
      Returns the model url.
      java.lang.Class<O> getOutputClass()
      Returns the model output class.
      int getQueueSize()
      Returns the configured size of the workers queue.
      static java.util.Properties getServingProperties​(java.nio.file.Path modelDir)
      Loads the serving properties from model folder.
      ModelInfo.Status getStatus()
      Returns the model loading status.
      java.lang.String getVersion()
      Returns the model version.
      int hashCode()
      void hasInputOutputClass​(java.lang.Class<I> inputClass, java.lang.Class<O> outputClass)
      Clarifies the input and output class when not specified.
      static java.lang.String inferDeviceName​(java.lang.String modelUrl)
      Infers which device to load.
      static java.lang.String inferEngine​(java.nio.file.Path modelDir, java.lang.String modelName)
      Infers engine name from model directory.
      static java.lang.String inferEngineFromUrl​(java.lang.String modelUrl)
      Infers engine name from model URL.
      static java.lang.String inferModelNameFromUrl​(java.lang.String url)
      Infer model name form model URL in case model name is not provided.
      void load​(ai.djl.Device device)
      Loads the model to the specified device.
      void setBatchSize​(int batchSize)
      Sets the configured batch size.
      void setMaxBatchDelayMillis​(int maxBatchDelayMillis)
      Sets the maximum delay in milliseconds to aggregate a batch.
      void setMaxIdleSeconds​(int maxIdleSeconds)
      Sets the configured max idle time in seconds of workers.
      void setModelId​(java.lang.String id)
      Sets the model ID.
      void setQueueSize​(int queueSize)
      Sets the configured size of the workers queue.
      java.lang.String toString()
      ai.djl.Device withDefaultDevice​(java.lang.String deviceName)
      Returns the default device for this model if device is null.

      • Methods inherited from class java.lang.Object

        clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • ModelInfo

        public ModelInfo​(java.lang.String modelUrl,
                 java.lang.Class<I> inputClass,
                 java.lang.Class<O> outputClass)
        Constructs a new ModelInfo instance.
        Parameters:
        inputClass - the model input class
        outputClass - the model output class
        modelUrl - the model Url

      • ModelInfo

        public ModelInfo​(java.lang.String id,
                 ai.djl.repository.zoo.Criteria<I,​O> criteria)
        Constructs a ModelInfo based on a Criteria.
        Parameters:
        id - the id for the created ModelInfo
        criteria - the model criteria

      • ModelInfo

        public ModelInfo​(java.lang.String id,
                 java.lang.String modelUrl,
                 java.lang.String version,
                 java.lang.String engineName,
                 java.lang.Class<I> inputClass,
                 java.lang.Class<O> outputClass,
                 int queueSize,
                 int maxIdleSeconds,
                 int maxBatchDelayMillis,
                 int batchSize)
        Constructs a new ModelInfo instance.
        Parameters:
        id - the ID of the model that will be used by workflow
        modelUrl - the model url
        version - the version of the model
        engineName - the engine to load the model
        inputClass - the model input class
        outputClass - the model output class
        queueSize - the maximum request queue size
        maxIdleSeconds - the initial maximum idle time for workers.
        maxBatchDelayMillis - the initial maximum delay when scaling up before giving up.
        batchSize - the batch size for this model.

    • Method Detail

      • load

        public void load​(ai.djl.Device device)
          throws ai.djl.ModelException,
                 java.io.IOException
        Loads the model to the specified device.
        Parameters:
        device - the device to load model on
        Throws:
        java.io.IOException - if failed to read model file
        ai.djl.ModelException - if failed to load the specified model

      • getModels

        public java.util.Map<ai.djl.Device,​ai.djl.repository.zoo.ZooModel<I,​O>> getModels()
        Returns all loaded models.
        Returns:
        all loaded models

      • getModel

        public ai.djl.repository.zoo.ZooModel<I,​O> getModel​(ai.djl.Device device)
        Returns the loaded ZooModel for a device.
        Parameters:
        device - the device to return the model on
        Returns:
        the loaded ZooModel

      • setModelId

        public void setModelId​(java.lang.String id)
        Sets the model ID.
        Parameters:
        id - the model ID

      • getModelId

        public java.lang.String getModelId()
        Returns the model ID.
        Returns:
        the model ID

      • getVersion

        public java.lang.String getVersion()
        Returns the model version.
        Returns:
        the model version

      • getEngineName

        public java.lang.String getEngineName()
        Returns the engine name.
        Returns:
        the engine name

      • getModelUrl

        public java.lang.String getModelUrl()
        Returns the model url.
        Returns:
        the model url

      • getStatus

        public ModelInfo.Status getStatus()
        Returns the model loading status.
        Returns:
        the model loading status

      • getInputClass

        public java.lang.Class<I> getInputClass()
        Returns the model input class.
        Returns:
        the model input class

      • getOutputClass

        public java.lang.Class<O> getOutputClass()
        Returns the model output class.
        Returns:
        the model output class

      • hasInputOutputClass

        public void hasInputOutputClass​(java.lang.Class<I> inputClass,
                                java.lang.Class<O> outputClass)
        Clarifies the input and output class when not specified.

        Warning: This is intended for internal use with reflection.

        Parameters:
        inputClass - the model input class
        outputClass - the model output class

      • setMaxIdleSeconds

        public void setMaxIdleSeconds​(int maxIdleSeconds)
        Sets the configured max idle time in seconds of workers.
        Parameters:
        maxIdleSeconds - the configured max idle time in seconds of workers

      • getMaxIdleSeconds

        public int getMaxIdleSeconds()
        Returns the configured max idle time in seconds of workers.
        Returns:
        the max idle time in seconds

      • setBatchSize

        public void setBatchSize​(int batchSize)
        Sets the configured batch size.
        Parameters:
        batchSize - the configured batch size

      • getBatchSize

        public int getBatchSize()
        Returns the configured batch size.
        Returns:
        the configured batch size

      • setMaxBatchDelayMillis

        public void setMaxBatchDelayMillis​(int maxBatchDelayMillis)
        Sets the maximum delay in milliseconds to aggregate a batch.
        Parameters:
        maxBatchDelayMillis - the maximum delay in milliseconds to aggregate a batch

      • getMaxBatchDelayMillis

        public int getMaxBatchDelayMillis()
        Returns the maximum delay in milliseconds to aggregate a batch.
        Returns:
        the maximum delay in milliseconds to aggregate a batch

      • setQueueSize

        public void setQueueSize​(int queueSize)
        Sets the configured size of the workers queue.
        Parameters:
        queueSize - the configured size of the workers queue

      • getQueueSize

        public int getQueueSize()
        Returns the configured size of the workers queue.
        Returns:
        requested size of the workers queue.

      • close

        public void close()
        Close all loaded models.

      • inferModelNameFromUrl

        public static java.lang.String inferModelNameFromUrl​(java.lang.String url)
        Infer model name form model URL in case model name is not provided.
        Parameters:
        url - the model URL
        Returns:
        the model name

      • inferEngineFromUrl

        public static java.lang.String inferEngineFromUrl​(java.lang.String modelUrl)
        Infers engine name from model URL.
        Parameters:
        modelUrl - the model URL
        Returns:
        the engine name

      • inferDeviceName

        public static java.lang.String inferDeviceName​(java.lang.String modelUrl)
        Infers which device to load.
        Parameters:
        modelUrl - the model URL
        Returns:
        the device name

      • inferEngine

        public static java.lang.String inferEngine​(java.nio.file.Path modelDir,
                                           java.lang.String modelName)
        Infers engine name from model directory.
        Parameters:
        modelDir - the model directory
        modelName - the model name
        Returns:
        the engine name

      • withDefaultDevice

        public ai.djl.Device withDefaultDevice​(java.lang.String deviceName)
        Returns the default device for this model if device is null.
        Parameters:
        deviceName - the device to use if it is not null
        Returns:
        a non-null device

      • downloadModel

        public static ai.djl.util.Pair<java.lang.String,​java.nio.file.Path> downloadModel​(java.lang.String modelUrl)
                                                                                 throws java.io.IOException
        Downloads model from the model URL.
        Parameters:
        modelUrl - the model URL
        Returns:
        model name and downloaded model path
        Throws:
        java.io.IOException - if failed to download the model

      • getServingProperties

        public static java.util.Properties getServingProperties​(java.nio.file.Path modelDir)
        Loads the serving properties from model folder.
        Parameters:
        modelDir - model directory
        Returns:
        the serving properties

      • equals

        public boolean equals​(java.lang.Object o)
        Overrides:
        equals in class java.lang.Object

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class java.lang.Object

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object