Package ai.vectros.resources.inference

Class AsyncInferenceClient

java.lang.Object

ai.vectros.resources.inference.AsyncInferenceClient

public class AsyncInferenceClient
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
protected final ClientOptions	clientOptions

Constructor Summary

Constructors

Constructor	Description
AsyncInferenceClient(ClientOptions clientOptions)

Method Summary

Modifier and Type	Method	Description
CompletableFuture<Iterable<ChatStreamEvent>>	chatInference(ChatRequest request)	Streams a model response as Server-Sent Events (SSE).
CompletableFuture<Iterable<ChatStreamEvent>>	chatInference(ChatRequest request, RequestOptions requestOptions)	Streams a model response as Server-Sent Events (SSE).
CompletableFuture<Iterable<DocumentAskStreamEvent>>	documentAsk(String id, DocumentAskRequest request)	Loads a single document's extracted text, supplies it as context, and streams a model answer about it.
CompletableFuture<Iterable<DocumentAskStreamEvent>>	documentAsk(String id, DocumentAskRequest request, RequestOptions requestOptions)	Loads a single document's extracted text, supplies it as context, and streams a model answer about it.
CompletableFuture<ModelsResponse>	listInferenceModels()	Returns every inference model available to you, including each model's context window, the plan tiers it's available on, and the exact credit rates charged per 1K input and output tokens.
CompletableFuture<ModelsResponse>	listInferenceModels(RequestOptions requestOptions)	Returns every inference model available to you, including each model's context window, the plan tiers it's available on, and the exact credit rates charged per 1K input and output tokens.
CompletableFuture<Iterable<RagStreamEvent>>	ragInference(RagRequest request)	Runs hybrid search over your indexed content, then streams a model answer grounded in the top results.
CompletableFuture<Iterable<RagStreamEvent>>	ragInference(RagRequest request, RequestOptions requestOptions)	Runs hybrid search over your indexed content, then streams a model answer grounded in the top results.
AsyncRawInferenceClient	withRawResponse()	Get responses with HTTP metadata like headers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

clientOptions

protected final ClientOptions clientOptions

Constructor Details

AsyncInferenceClient

public AsyncInferenceClient(ClientOptions clientOptions)

Method Details

withRawResponse

public AsyncRawInferenceClient withRawResponse()

Get responses with HTTP metadata like headers

listInferenceModels

public CompletableFuture<ModelsResponse> listInferenceModels()

Returns every inference model available to you, including each model's context window, the plan tiers it's available on, and the exact credit rates charged per 1K input and output tokens. Use this to populate model pickers and to validate a request before calling /v1/chat, /v1/rag, or /v1/documents/{id}/ask.

listInferenceModels

public CompletableFuture<ModelsResponse> listInferenceModels(RequestOptions requestOptions)

Returns every inference model available to you, including each model's context window, the plan tiers it's available on, and the exact credit rates charged per 1K input and output tokens. Use this to populate model pickers and to validate a request before calling /v1/chat, /v1/rag, or /v1/documents/{id}/ask.

chatInference

public CompletableFuture<Iterable<ChatStreamEvent>> chatInference(ChatRequest request)

Streams a model response as Server-Sent Events (SSE). Send the full conversation history in the messages array; a message with role system is extracted and used as the system prompt. Token cost is debited from your pre-paid inference balance (in cents), and a small per-call flat fee is debited from your monthly platform credit allowance. Requires the inference:r scope.

chatInference

public CompletableFuture<Iterable<ChatStreamEvent>> chatInference(ChatRequest request,
 RequestOptions requestOptions)

Streams a model response as Server-Sent Events (SSE). Send the full conversation history in the messages array; a message with role system is extracted and used as the system prompt. Token cost is debited from your pre-paid inference balance (in cents), and a small per-call flat fee is debited from your monthly platform credit allowance. Requires the inference:r scope.

documentAsk

public CompletableFuture<Iterable<DocumentAskStreamEvent>> documentAsk(String id,
 DocumentAskRequest request)

Loads a single document's extracted text, supplies it as context, and streams a model answer about it. The document must be fully indexed. If the document exceeds the 32K-token cap, the call returns 413 with no credits charged — use POST /v1/rag instead to answer over larger or multi-document collections. Requires the inference:r scope.

documentAsk

public CompletableFuture<Iterable<DocumentAskStreamEvent>> documentAsk(String id,
 DocumentAskRequest request,
 RequestOptions requestOptions)

Loads a single document's extracted text, supplies it as context, and streams a model answer about it. The document must be fully indexed. If the document exceeds the 32K-token cap, the call returns 413 with no credits charged — use POST /v1/rag instead to answer over larger or multi-document collections. Requires the inference:r scope.

ragInference

public CompletableFuture<Iterable<RagStreamEvent>> ragInference(RagRequest request)

Runs hybrid search over your indexed content, then streams a model answer grounded in the top results. The SSE stream emits a search_results event first (carrying the matched results and their metadata), an optional truncation_warning if some retrieved results were dropped before the prompt was built — either because they didn't fit the model's context window or because a result had no groundable text to include — then content_delta chunks, and finally a terminal done event. Requires the inference:r scope.

ragInference

public CompletableFuture<Iterable<RagStreamEvent>> ragInference(RagRequest request,
 RequestOptions requestOptions)

Runs hybrid search over your indexed content, then streams a model answer grounded in the top results. The SSE stream emits a search_results event first (carrying the matched results and their metadata), an optional truncation_warning if some retrieved results were dropped before the prompt was built — either because they didn't fit the model's context window or because a result had no groundable text to include — then content_delta chunks, and finally a terminal done event. Requires the inference:r scope.