Package ai.pipestream.common.util

Class ProcessingBufferImpl<T extends com.google.protobuf.Message>

java.lang.Object
ai.pipestream.common.util.ProcessingBufferImpl<T>
Type Parameters:
T - the type of protobuf message to store
All Implemented Interfaces:
ProcessingBuffer<T>
public class ProcessingBufferImpl<T extends com.google.protobuf.Message> extends Object implements ProcessingBuffer<T>
Implementation of the ProcessingBuffer interface. This class uses a bounded queue to store protobuf messages and provides methods to save them to disk. It is thread-safe and clones the protobufs to prevent memory reference issues.

  • Constructor Summary

    Constructors
    Constructor
    Description
    ProcessingBufferImpl(int capacity, Class<T> messageClass)
    Creates a new ProcessingBufferImpl with the specified capacity and message class.
    ProcessingBufferImpl(int capacity, Class<T> messageClass, FileSystem fileSystem)
    Creates a new ProcessingBufferImpl with the specified capacity, message class, and file system.
    ProcessingBufferImpl(int capacity, Class<T> messageClass, FileSystem fileSystem, Path saveLocation, String savePrefix, int savePrecision)
    Creates a new ProcessingBufferImpl with the specified capacity, message class, file system, and save parameters.
    ProcessingBufferImpl(int capacity, Class<T> messageClass, Path saveLocation, String savePrefix, int savePrecision)
    Creates a new ProcessingBufferImpl with the specified capacity, message class, and save parameters.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    add(T message)
    Adds a message to the buffer.
    void
    clear()
    Clears all messages from the buffer.
    void
    saveOnShutdown()
    Saves the buffer contents to disk when the bean is destroyed.
    void
    saveToDisk()
    Saves all messages in the buffer to disk using default parameters.
    void
    saveToDisk(String prefix, int numberPrecision)
    Saves all messages in the buffer to disk to "
    void
    saveToDisk(Path location, String fileNamePrefix, int numberPrecision)
    Saves all messages in the buffer to disk.
    int
    size()
    Returns the number of messages in the buffer.
    List<T>
    snapshot()
    Returns a snapshot of all items currently in the buffer.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • ProcessingBufferImpl

      public ProcessingBufferImpl(int capacity, Class<T> messageClass, FileSystem fileSystem, Path saveLocation, String savePrefix, int savePrecision)
      Creates a new ProcessingBufferImpl with the specified capacity, message class, file system, and save parameters.
      Parameters:
      capacity - the maximum number of messages to store
      messageClass - the class of the protobuf message
      fileSystem - the file system to use for saving files
      saveLocation - the directory to save the files to
      savePrefix - the prefix to use for the file names
      savePrecision - the number of digits to use for the file name suffix

    • ProcessingBufferImpl

      public ProcessingBufferImpl(int capacity, Class<T> messageClass, Path saveLocation, String savePrefix, int savePrecision)
      Creates a new ProcessingBufferImpl with the specified capacity, message class, and save parameters. Uses the default file system.
      Parameters:
      capacity - the maximum number of messages to store
      messageClass - the class of the protobuf message
      saveLocation - the directory to save the files to
      savePrefix - the prefix to use for the file names
      savePrecision - the number of digits to use for the file name suffix

    • ProcessingBufferImpl

      public ProcessingBufferImpl(int capacity, Class<T> messageClass, FileSystem fileSystem)
      Creates a new ProcessingBufferImpl with the specified capacity, message class, and file system. Uses default save parameters.
      Parameters:
      capacity - the maximum number of messages to store
      messageClass - the class of the protobuf message
      fileSystem - the file system to use for saving files

    • ProcessingBufferImpl

      public ProcessingBufferImpl(int capacity, Class<T> messageClass)
      Creates a new ProcessingBufferImpl with the specified capacity and message class. Uses the default file system and default save parameters.
      Parameters:
      capacity - the maximum number of messages to store
      messageClass - the class of the protobuf message

  • Method Details

    • saveOnShutdown

      @PreDestroy public void saveOnShutdown()
      Saves the buffer contents to disk when the bean is destroyed. This method is called automatically during application shutdown. If the buffer is empty, nothing is saved.

    • add

      public void add(T message)
      Description copied from interface: ProcessingBuffer
      Adds a message to the buffer. If the buffer is full, the oldest message will be removed.
      Specified by:
      add in interface ProcessingBuffer<T extends com.google.protobuf.Message>
      Parameters:
      message - the message to add

    • saveToDisk

      public void saveToDisk()
      Description copied from interface: ProcessingBuffer
      Saves all messages in the buffer to disk using default parameters. Default location is the current directory, default prefix is "protobuf", and default precision is 3 digits.
      Specified by:
      saveToDisk in interface ProcessingBuffer<T extends com.google.protobuf.Message>

    • saveToDisk

      public void saveToDisk(String prefix, int numberPrecision)
      Description copied from interface: ProcessingBuffer
      Saves all messages in the buffer to disk to "
      ./
      " directory.
      Specified by:
      saveToDisk in interface ProcessingBuffer<T extends com.google.protobuf.Message>
      Parameters:
      prefix - the prefix to use for the file names
      numberPrecision - the number of digits to use for the file name suffix

    • saveToDisk

      public void saveToDisk(Path location, String fileNamePrefix, int numberPrecision)
      Description copied from interface: ProcessingBuffer
      Saves all messages in the buffer to disk.
      Specified by:
      saveToDisk in interface ProcessingBuffer<T extends com.google.protobuf.Message>
      Parameters:
      location - the directory to save the files to
      fileNamePrefix - the prefix to use for the file names
      numberPrecision - the number of digits to use for the file name suffix

    • size

      public int size()
      Description copied from interface: ProcessingBuffer
      Returns the number of messages in the buffer.
      Specified by:
      size in interface ProcessingBuffer<T extends com.google.protobuf.Message>
      Returns:
      the number of messages

    • clear

      public void clear()
      Description copied from interface: ProcessingBuffer
      Clears all messages from the buffer.
      Specified by:
      clear in interface ProcessingBuffer<T extends com.google.protobuf.Message>

    • snapshot

      public List<T> snapshot()
      Description copied from interface: ProcessingBuffer
      Returns a snapshot of all items currently in the buffer. This method creates a new list to prevent external modification of the buffer. The returned list represents the buffer contents at the moment of the call. This is useful for analysis and reporting.
      Specified by:
      snapshot in interface ProcessingBuffer<T extends com.google.protobuf.Message>
      Returns:
      a new list containing a snapshot of all messages in the buffer