Package ai.pipestream.module.pipelineprobe.pipelinecrawl

package ai.pipestream.module.pipelineprobe.pipelinecrawl

Related Packages

Package	Description
ai.pipestream.module.pipelineprobe

Class	Description
AccountManager	Resolves the test account for a RunPipelineCrawl run.
ActiveCrawlRegistry	Tracks the crawl_ids of pipeline-crawl runs that currently have an open observation stream (an in-flight OrchestrationDriver run).
CrawlCleanupService	Ninja cleanup for pipeline-crawl runs: by default a run deletes every resource it provisioned, so an hourly health-check e2e leaves the platform exactly as it found it.
CrawlCleanupService.RunResources	Everything one run needs torn down.
DatasourceManager	Creates a datasource for a RunPipelineCrawl run and returns its id + api key directly.
DatasourceManager.DatasourceCredentials	Credentials for a freshly-created datasource.
DefaultChunkerConfig	Builds the chunker module's custom_config for a pipeline-crawl run.
EmbedderFixturesRegistrar	Idempotent registration of the chunker / embedder / VectorSet fixtures the chunker→embedder→sink Pipeline tab shape needs before its IndexPlan can provision KNN fields.
EventEmitter	Wraps the gRPC response observer with the boilerplate common to every progress event: serialization on the observer, Timestamp build, pretty step name lookup.
GraphBinder	Binds a datasource to an active graph at its entry node, returning the datasource-instance id directly.
GrpcRetry	Bounded retry wrapper around a blocking gRPC stub call.
GrpcRetry.RetryPolicy	Retry budget for a single op.
IndexingLedgerPoller	Polls repository-service's indexing ledger for per-crawl receipt progress.
IndexingLedgerPoller.CrawlProgress	Receipt counts for one crawl, partitioned the way the run report wants them.
JdbcCrawlManager	Drives the JDBC source for a RunPipelineCrawl run: Create the crawl definition via gRPC.
JdbcCrawlManager.JdbcDispatch	Result of JdbcCrawlManager.setupAndTrigger(String, String, String, JdbcSourceConfig): the connector's triggerId (also the crawl_id on every dispatched PipeStream) and the crawlName the connector now has registered.
JdbcDispatchSubscriber	Subscribes to the JDBC connector's streamCrawlStatus server-streaming RPC and hydrates a RunLedger from the per-event payload.
JdbcDispatchSubscriber.Subscription	The active subscription.
OpenSearchPoller	Polls opensearch-manager for per-crawl indexed-doc counts across the whole index family (base + --chunk--* + --vs--*).
OpenSearchPoller.FamilySnapshot	One tick's view of the crawl's index family.
OrchestrationDriver	The actual pipeline-crawl orchestration body (preferred harness path — blocking gRPC, no Mutiny).
OrchestrationDriver.RunDeadlineExceeded	Hard wall-clock deadline exceeded.
OrchestrationDriver.RunInterrupted	Wait loop interrupted (sidecar shutdown, etc).
OrchestrationDriver.RunOsDrainFailed	The OS-drain phase found docs the platform terminally failed to index.
OrchestrationDriver.RunSubscriptionFailed	Dispatch-stream subscription terminated with a real failure: either the connector emitted a malformed event our handler refused to accept, or the gRPC stream errored with a non-CANCELLED status.
PipeDocOutputSummary	Renders a final PipeDoc into a readable Struct for the UI's single-doc output panel.
PipelineCrawlerImpl	gRPC entry point for RunPipelineCrawl.
PipelineCrawlerResource	REST/SSE bridge from the FE's Pipeline tab to the gRPC PipelineCrawlerImpl.
PipelineCrawlerResource.PipelineCrawlPayload	FE-facing payload.
PipelineCrawlerResource.PipelineCrawlPayload.JdbcPayload	JDBC source payload.
PipelineCrawlerResource.PipelineCrawlPayload.S3Payload	S3 source payload.
PipelineCrawlerResource.PipelineCrawlPayload.SingleDocPayload	SINGLE_DOC source payload.
PipelineEventsConsumer	Live observation feed for the Pipeline-tab panels.
RawUploadRestClient	Streaming HTTP client to connector-intake's raw upload endpoint (POST /uploads/raw) — the S3-style path that streams a document of any length straight to :intake/S3 without ever buffering it whole.
RunLedger	Single source of truth for one pipeline-crawl run's progress.
RunRedisStreamsCleanup	Drops a run's per-graph redis streams once the pipeline-crawl finishes.
RunRegistry	Owns the lifetime of pipeline-crawl runs INDEPENDENTLY of any transport connection.
RunRegistry.RunHandle	One run: its thread, journal, and subscribers.
RunRegistry.RunInfo	Summary row for one run.
RunRegistry.RunState	Run state for the runs listing.
S3CrawlManager	Drives the S3 source for a RunPipelineCrawl run — the analogue of JdbcCrawlManager for buckets.
S3CrawlManager.S3Dispatch	Result of S3CrawlManager.hydrateAndTrigger(String, String, String, S3SourceConfig): the connector's request id (the crawl's idempotency key) and how many docs were uploaded during hydration (0 when hydration was skipped).
S3DispatchSubscriber	Subscribes to the s3-connector's StreamCrawlStatus for one crawl and mirrors its state into a S3DispatchSubscriber.S3DispatchState the orchestrator's wait loop reads each tick — the S3 counterpart of JdbcDispatchSubscriber, but count-based: the connector reports how many objects it dispatched (and, at COMPLETED, the final total) rather than per-doc ids.
S3DispatchSubscriber.S3DispatchState	Live mirror of one crawl's connector-side state.
S3SampleDocLoader	Resolves the set of documents to upload into S3 before an S3 pipeline-crawl run, when hydrate_sample_data is set.
S3SampleDocLoader.Sample	One document to upload: its object key (path within the corpus), bytes, mime type.
SidecarTapPoller	Reads in-process tap counters and rich-observation snapshots for the orchestrator's poll loop.
SingleDocDispatcher	Drives the SINGLE_DOC source for a RunPipelineCrawl run.
SingleDocDispatcher.SingleDocDispatch	Result of SingleDocDispatcher.uploadAndDispatch(String, String, String, SingleDocSourceConfig, String): the doc_id the upload produced.
StageLedger	One stage's receive-side view in a RunLedger: which doc ids landed at this stage's tap, and how many we expect.
TerminalDocCapture	Captures the most recent PipeDoc that reached the terminal sidecar node, keyed by account.