Class SampleDataLoader

java.lang.Object

ai.pipestream.module.pipelineprobe.sampledata.SampleDataLoader

@ApplicationScoped
public class SampleDataLoader
extends Object

Reads the per-doc PipeDocs out of a registered sample-data set.

Pairs with SampleDataRegistry: the registry says where a step's input docs live (fixtures/court/<step>/doc_NNNN.pb.gz) and how many there are; this loads them off the classpath, gunzipping each file (one non-delimited PipeDoc per .pb.gz). Lazy and single-doc — a 1000-doc stress run never holds more than one doc's bytes at a time.

Constructor Summary

Constructors

Constructor	Description
SampleDataLoader()	Creates the sample-data loader.

Method Summary

Modifier and Type	Method	Description
boolean	hasCorpusFor(String stage)	True if a dataset feeds the given step (i.e.
Optional<ai.pipestream.data.v1.PipeDoc>	loadDoc(String stage, int index)	Load the index-th (1-based) doc of the dataset that feeds stage.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SampleDataLoader

public SampleDataLoader()

Creates the sample-data loader. Instances are managed by CDI.

Method Details

hasCorpusFor

public boolean hasCorpusFor(String stage)

True if a dataset feeds the given step (i.e. /corpus can run it).

Parameters:

stage - the pipeline step/stage to check for a registered dataset

Returns:

true if a dataset feeds the given step, false otherwise

loadDoc

public Optional<ai.pipestream.data.v1.PipeDoc> loadDoc(String stage,
 int index)

Load the index-th (1-based) doc of the dataset that feeds stage. Empty if no such dataset or no doc at that index.

Parameters:

stage - the pipeline step/stage whose dataset to read from

index - the 1-based index of the doc to load

Returns:

the loaded doc, or empty when no dataset feeds the stage or no doc exists at the index