Package ai.djl.basicdataset.utils

Class TextData.Configuration

java.lang.Object
ai.djl.basicdataset.utils.TextData.Configuration
Enclosing class:
TextData
public static final class TextData.Configuration extends Object
The configuration for creating a TextData value in a Dataset.

  • Constructor Details

    • Configuration

      public Configuration()

  • Method Details

    • setTextProcessors

      public TextData.Configuration setTextProcessors(List<ai.djl.modality.nlp.preprocess.TextProcessor> textProcessors)
      Sets the TextProcessors to use for the text data.
      Parameters:
      textProcessors - the TextProcessors
      Returns:
      this configuration

    • setTextEmbedding

      public TextData.Configuration setTextEmbedding(ai.djl.modality.nlp.embedding.TextEmbedding textEmbedding)
      Sets the TextEmbedding to use to embed the text data.
      Parameters:
      textEmbedding - the TextEmbedding
      Returns:
      this configuration

    • setVocabulary

      public TextData.Configuration setVocabulary(ai.djl.modality.nlp.Vocabulary vocabulary)
      Sets the Vocabulary to use to hold the text data.
      Parameters:
      vocabulary - the Vocabulary
      Returns:
      this configuration

    • setEmbeddingSize

      public TextData.Configuration setEmbeddingSize(int embeddingSize)
      Sets the size for new TextEmbeddings.
      Parameters:
      embeddingSize - the embedding size
      Returns:
      this configuration

    • setUnknownToken

      public TextData.Configuration setUnknownToken(String unknownToken)
      Sets the default unknown token.
      Parameters:
      unknownToken - the String value of unknown token
      Returns:
      this configuration

    • setReservedTokens

      public TextData.Configuration setReservedTokens(List<String> reservedTokens)
      Sets the list of reserved tokens.
      Parameters:
      reservedTokens - true to train the text embedding
      Returns:
      this configuration

    • update

      public TextData.Configuration update(TextData.Configuration other)
      Updates this TextData.Configuration with the non-null values from another configuration.
      Parameters:
      other - the other configuration to use to update this
      Returns:
      this configuration after updating