Package ai.vectros.resources.inference

Class InferenceClient

java.lang.Object
ai.vectros.resources.inference.InferenceClient
public class InferenceClient extends Object

  • Field Details

  • Constructor Details

    • InferenceClient

      public InferenceClient(ClientOptions clientOptions)

  • Method Details

    • withRawResponse

      public RawInferenceClient withRawResponse()
      Get responses with HTTP metadata like headers

    • listInferenceModels

      public ModelsResponse listInferenceModels()
      Returns every inference model available to you, including each model's context window, the plan tiers it's available on, and the exact credit rates charged per 1K input and output tokens. Use this to populate model pickers and to validate a request before calling /v1/chat, /v1/rag, or /v1/documents/{id}/ask.

    • listInferenceModels

      public ModelsResponse listInferenceModels(RequestOptions requestOptions)
      Returns every inference model available to you, including each model's context window, the plan tiers it's available on, and the exact credit rates charged per 1K input and output tokens. Use this to populate model pickers and to validate a request before calling /v1/chat, /v1/rag, or /v1/documents/{id}/ask.

    • chatInference

      public Iterable<ChatStreamEvent> chatInference(ChatRequest request)
      Streams a model response as Server-Sent Events (SSE). Send the full conversation history in the messages array; a message with role system is extracted and used as the system prompt. Token cost is debited from your pre-paid inference balance (in cents), and a small per-call flat fee is debited from your monthly platform credit allowance. Requires the inference:r scope.

    • chatInference

      public Iterable<ChatStreamEvent> chatInference(ChatRequest request, RequestOptions requestOptions)
      Streams a model response as Server-Sent Events (SSE). Send the full conversation history in the messages array; a message with role system is extracted and used as the system prompt. Token cost is debited from your pre-paid inference balance (in cents), and a small per-call flat fee is debited from your monthly platform credit allowance. Requires the inference:r scope.

    • documentAsk

      public Iterable<DocumentAskStreamEvent> documentAsk(String id, DocumentAskRequest request)
      Loads a single document's extracted text, supplies it as context, and streams a model answer about it. The document must be fully indexed. If the document exceeds the 32K-token cap, the call returns 413 with no credits charged — use POST /v1/rag instead to answer over larger or multi-document collections. Requires the inference:r scope.

    • documentAsk

      public Iterable<DocumentAskStreamEvent> documentAsk(String id, DocumentAskRequest request, RequestOptions requestOptions)
      Loads a single document's extracted text, supplies it as context, and streams a model answer about it. The document must be fully indexed. If the document exceeds the 32K-token cap, the call returns 413 with no credits charged — use POST /v1/rag instead to answer over larger or multi-document collections. Requires the inference:r scope.

    • ragInference

      public Iterable<RagStreamEvent> ragInference(RagRequest request)
      Runs hybrid search over your indexed content, then streams a model answer grounded in the top results. The SSE stream emits a search_results event first (carrying the matched results and their metadata), an optional truncation_warning if some retrieved results were dropped before the prompt was built — either because they didn't fit the model's context window or because a result had no groundable text to include — then content_delta chunks, and finally a terminal done event. Requires the inference:r scope.

    • ragInference

      public Iterable<RagStreamEvent> ragInference(RagRequest request, RequestOptions requestOptions)
      Runs hybrid search over your indexed content, then streams a model answer grounded in the top results. The SSE stream emits a search_results event first (carrying the matched results and their metadata), an optional truncation_warning if some retrieved results were dropped before the prompt was built — either because they didn't fit the model's context window or because a result had no groundable text to include — then content_delta chunks, and finally a terminal done event. Requires the inference:r scope.