Class ParserScaffold

java.lang.Object
ai.pipestream.module.pipelineprobe.moduletest.scaffold.ParserScaffold
All Implemented Interfaces:
StepScaffold
public class ParserScaffold extends Object implements StepScaffold
Parser scaffold: the parser has no captured <step>-input-court dataset — its input is the raw source file, which exists BEFORE any capture tap (and the court corpus is JDBC, which skips the parser entirely). So this scaffold supplies its own corpus: the mixed-format sample-doc-types raw corpus (pdf/office/epub/html/email/…) that the e2e S3 crawl hydrates into the bucket, read via the SAME S3SampleDocLoader — but handed straight to the door instead of going through S3 + crawl.

Each raw file is wrapped the way connector-intake's gRPC upload wraps it: a PipeDoc carrying the bytes in its BlobBag — exactly the shape the parser hydrates and parses in production.

Samples are resolved once per session (prepare()) and held in memory — same footprint the e2e hydration path already pays (the corpus is capped at max-sample-docs, ~250 files).

  • Constructor Details

    • ParserScaffold

      public ParserScaffold(S3SampleDocLoader sampleLoader)
      Creates the parser scaffold.
      Parameters:
      sampleLoader - the loader used to resolve the raw sample-doc-types corpus

  • Method Details

    • module

      public String module()
      Description copied from interface: StepScaffold
      The module/step this scaffold targets (e.g. "chunker").
      Specified by:
      module in interface StepScaffold
      Returns:
      the module/step name this scaffold targets

    • prepare

      public void prepare()
      Description copied from interface: StepScaffold
      Provision external state (vector sets, fresh index, …). Called once per session.
      Specified by:
      prepare in interface StepScaffold

    • corpusSize

      public OptionalInt corpusSize()
      Description copied from interface: StepScaffold
      Number of docs in the scaffold-supplied corpus; empty = use the fixture-jar dataset.
      Specified by:
      corpusSize in interface StepScaffold
      Returns:
      the scaffold-supplied corpus size, or empty to use the fixture-jar dataset

    • corpusDoc

      public Optional<ai.pipestream.data.v1.PipeDoc> corpusDoc(int index)
      Description copied from interface: StepScaffold
      The scaffold-supplied corpus doc at 1-based index; empty when out of range. Only meaningful when StepScaffold.corpusSize() is present.
      Specified by:
      corpusDoc in interface StepScaffold
      Parameters:
      index - the 1-based index of the corpus doc to fetch
      Returns:
      the corpus doc at index, or empty when out of range

    • corpusName

      public String corpusName()
      Description copied from interface: StepScaffold
      Display name of the scaffold-supplied corpus (e.g. the source jar).
      Specified by:
      corpusName in interface StepScaffold
      Returns:
      the corpus display name, or null when there is no scaffold-supplied corpus

    • wrap

      public ai.pipestream.data.v1.PipeStream wrap(ai.pipestream.data.v1.PipeDoc doc)
      Description copied from interface: StepScaffold
      Turn a corpus doc into the step's expected input PipeStream (directives, context_params, …).
      Specified by:
      wrap in interface StepScaffold
      Parameters:
      doc - the corpus document to wrap
      Returns:
      the PipeStream shaped as the step expects

    • cleanup

      public void cleanup()
      Description copied from interface: StepScaffold
      Tear down anything StepScaffold.prepare() created (e.g. drop the fresh index).
      Specified by:
      cleanup in interface StepScaffold