Class AppendDataStateData
- java.lang.Object
-
- edu.stanford.slac.archiverappliance.PlainPB.AppendDataStateData
-
public class AppendDataStateData extends Object
Companion class to PlainPBStoragePlugin that handles the appending of event streams in a partition-aware fashion. This is used both by the engine and by ETL.- Author:
- mshankar
-
-
Constructor Summary
Constructors Constructor Description AppendDataStateData(PartitionGranularity partitionGranularity, String rootFolder, String desc, Timestamp lastKnownTimestamp, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
-
Method Summary
Modifier and Type Method Description boolean
bulkAppend(String pvName, ETLContext context, ETLBulkStream bulkStream, String extension, String extensionToCopyFrom)
Append data in bulk skipping some of the per event checks.int
partitionBoundaryAwareAppendData(BasicContext context, String pvName, EventStream stream, String extension, String extensionToCopyFrom)
Append data into PB files honoring partition boundaries switching into new partitions as we cross the boundary.
-
-
-
Constructor Detail
-
AppendDataStateData
public AppendDataStateData(PartitionGranularity partitionGranularity, String rootFolder, String desc, Timestamp lastKnownTimestamp, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
- Parameters:
partitionGranularity
- partitionGranularity of the PB plugin.rootFolder
- RootFolder of the PB plugindesc
- Desc for logging purposeslastKnownTimestamp
- This is probably the most important argument here. This is the last known timestamp in this storage. If null, we assume time(0) for the last known timestamp.compressionMode
- CompressionModepv2key
- PVNameToKeyMapping
-
-
Method Detail
-
partitionBoundaryAwareAppendData
public int partitionBoundaryAwareAppendData(BasicContext context, String pvName, EventStream stream, String extension, String extensionToCopyFrom) throws IOException
Append data into PB files honoring partition boundaries switching into new partitions as we cross the boundary.- We make sure timestamp monotonicity is maintained.
- We generate clean partitions.
- Parameters:
context
-pvName
- The PV namestream
-extension
-extensionToCopyFrom
-- Returns:
- eventsAppended
- Throws:
IOException
-
-
bulkAppend
public boolean bulkAppend(String pvName, ETLContext context, ETLBulkStream bulkStream, String extension, String extensionToCopyFrom) throws IOException
Append data in bulk skipping some of the per event checks.- Parameters:
pvName
- The PV namecontext
- The ETL contextbulkStream
- The ETL bulk streamextension
-extensionToCopyFrom
-- Returns:
- boolean
- Throws:
IOException
-
-
-