| Package | Description |
|---|---|
| xyz.felh.openai.jtokkit.api | |
| xyz.felh.openai.jtokkit.utils |
| Modifier and Type | Method and Description |
|---|---|
Encoding |
EncodingRegistry.getEncoding(EncodingType encodingType)
Returns the encoding with the given type.
|
Encoding |
EncodingRegistry.getEncodingForModel(ModelType modelType)
Returns the encoding that is used for the given model type.
|
| Modifier and Type | Method and Description |
|---|---|
java.util.Optional<Encoding> |
EncodingRegistry.getEncoding(java.lang.String encodingName)
Returns the encoding with the given name, if it exists.
|
java.util.Optional<Encoding> |
EncodingRegistry.getEncodingForModel(java.lang.String modelName)
Returns the encoding that is used for the given model type, if it exists.
|
| Modifier and Type | Method and Description |
|---|---|
EncodingRegistry |
EncodingRegistry.registerCustomEncoding(Encoding encoding)
Registers a new custom encoding with the given name.
|
| Modifier and Type | Method and Description |
|---|---|
static Encoding |
TikTokenUtils.getEncoding(EncodingType encodingType)
获取一个Encoding对象,通过Encoding类型
|
static Encoding |
TikTokenUtils.getEncoding(java.lang.String modelName)
获取一个Encoding对象,通过模型名称
|
| Modifier and Type | Method and Description |
|---|---|
static java.lang.String |
TikTokenUtils.decode(Encoding enc,
java.util.List<java.lang.Integer> encoded)
通过Encoding和encoded数组反推text信息
|
static java.util.List<java.lang.Integer> |
TikTokenUtils.encode(Encoding enc,
java.lang.String text)
通过Encoding和text获取编码数组
|
static int |
TikTokenUtils.tokens(Encoding enc,
java.lang.String text)
通过Encoding计算text信息的tokens
|
Copyright © 2023. All Rights Reserved.