Uses of Interface
xyz.felh.openai.gpt3.tokenizer.ByteSequence

Uses of ByteSequence in xyz.felh.openai.gpt3.tokenizer

Classes in xyz.felh.openai.gpt3.tokenizer that implement ByteSequence

Modifier and Type	Class and Description
static class	ByteSequence.Of An immutable implementation of the ByteSequence.

Fields in xyz.felh.openai.gpt3.tokenizer declared as ByteSequence

Modifier and Type	Field and Description
static ByteSequence	ByteSequence.EMPTY The empty ByteSequence of length 0.

Methods in xyz.felh.openai.gpt3.tokenizer that return ByteSequence

Modifier and Type	Method and Description
static ByteSequence	ByteSequence.copyOf(ByteSequence sequence) Returns an immutable ByteSequence that is a copy of the specified ByteSequence.
static ByteSequence	ByteSequence.from(java.lang.String text) Creates a ByteSequence from the specified String using the UTF-8 charset.
static ByteSequence	ByteSequence.from(java.lang.String text, java.nio.charset.Charset charset) Creates a ByteSequence from the specified String using the specified Charset.
static ByteSequence	ByteSequence.of(byte[] bytes) Returns a new ByteSequence instance containing the specified byte array.
ByteSequence	ByteSequence.subSequence(int start, int end) Returns a new ByteSequence that is a sub-sequence of the current byte sequence.

Methods in xyz.felh.openai.gpt3.tokenizer that return types with arguments of type ByteSequence

Modifier and Type	Method and Description
java.util.Map<ByteSequence,java.lang.Integer>	Encoding.mergeableRanks()

Methods in xyz.felh.openai.gpt3.tokenizer with parameters of type ByteSequence

Modifier and Type	Method and Description
protected int	GPT3Tokenizer.bytePairMerge(ByteSequence piece, java.util.Collection<java.lang.Integer> result)
static ByteSequence	ByteSequence.copyOf(ByteSequence sequence) Returns an immutable ByteSequence that is a copy of the specified ByteSequence.
protected int	GPT3Tokenizer.getRank(ByteSequence piece, java.util.List<xyz.felh.openai.gpt3.tokenizer.GPT3Tokenizer.IntPair> partsList, int startIdx)