Class AppendDataStateData


  • public class AppendDataStateData
    extends Object
    Companion class to PlainPBStoragePlugin that handles the appending of event streams in a partition-aware fashion. This is used both by the engine and by ETL.
    Author:
    mshankar
    • Constructor Detail

      • AppendDataStateData

        public AppendDataStateData​(PartitionGranularity partitionGranularity,
                                   String rootFolder,
                                   String desc,
                                   Timestamp lastKnownTimestamp,
                                   PlainPBStoragePlugin.CompressionMode compressionMode,
                                   PVNameToKeyMapping pv2key)
        Parameters:
        partitionGranularity - partitionGranularity of the PB plugin.
        rootFolder - RootFolder of the PB plugin
        desc - Desc for logging purposes
        lastKnownTimestamp - This is probably the most important argument here. This is the last known timestamp in this storage. If null, we assume time(0) for the last known timestamp.
        compressionMode - CompressionMode
        pv2key - PVNameToKeyMapping
    • Method Detail

      • partitionBoundaryAwareAppendData

        public int partitionBoundaryAwareAppendData​(BasicContext context,
                                                    String pvName,
                                                    EventStream stream,
                                                    String extension,
                                                    String extensionToCopyFrom)
                                             throws IOException
        Append data into PB files honoring partition boundaries switching into new partitions as we cross the boundary.
        1. We make sure timestamp monotonicity is maintained.
        2. We generate clean partitions.
        Parameters:
        context -  
        pvName - The PV name
        stream -  
        extension -  
        extensionToCopyFrom -  
        Returns:
        eventsAppended  
        Throws:
        IOException -  
      • bulkAppend

        public boolean bulkAppend​(String pvName,
                                  ETLContext context,
                                  ETLBulkStream bulkStream,
                                  String extension,
                                  String extensionToCopyFrom)
                           throws IOException
        Append data in bulk skipping some of the per event checks.
        Parameters:
        pvName - The PV name
        context - The ETL context
        bulkStream - The ETL bulk stream
        extension -  
        extensionToCopyFrom -  
        Returns:
        boolean  
        Throws:
        IOException -