-
public final class TaskRouterCompletely task-agnostic. For each request it finds (or lazy-loads) the adapter for the model, runs it, times it, and reports — for any inference kind that has a registered engine.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classTaskRouter.Companion
-
Constructor Summary
Constructors Constructor Description TaskRouter(Map<String, InferenceAdapter> adapters, Map<String, ModelConfig> configs, AdapterLoader loader)
-
Method Summary
Modifier and Type Method Description final List<TaskResult>handle(TaskMessage batch)-
-
Constructor Detail
-
TaskRouter
TaskRouter(Map<String, InferenceAdapter> adapters, Map<String, ModelConfig> configs, AdapterLoader loader)
-
-
Method Detail
-
handle
final List<TaskResult> handle(TaskMessage batch)
-
-
-
-