Class PlainPBPathNameUtility
- java.lang.Object
-
- edu.stanford.slac.archiverappliance.PlainPB.PlainPBPathNameUtility
-
public class PlainPBPathNameUtility extends Object
A utility class with a bunch of methods that operate on the path names used by the PlainPB storage plugin.- Author:
- mshankar
-
-
Constructor Summary
Constructors Constructor Description PlainPBPathNameUtility()
-
Method Summary
Modifier and Type Method Description static edu.stanford.slac.archiverappliance.PlainPB.PlainPBPathNameUtility.StartEndTimeFromName
determineTimesFromFileName(String pvName, String finalNameComponent, PartitionGranularity partitionGranularity, PVNameToKeyMapping pv2key)
Determines the times for a chunk simply from the file name.static Path[]
getAllPathsForPV(ArchPaths archPaths, String rootFolder, String pvName, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
This method returns all the paths that could contain data for a PV sorted according to the name (which in our case should translate to time).static Path
getMostRecentPathBeforeTime(ArchPaths archPaths, String rootFolder, String pvName, Timestamp startts, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
If a PV changes infrequently, we often will not have a sample in the given time frame.static Path
getPathNameForTime(PlainPBStoragePlugin plugin, String pvName, long epochSeconds, ArchPaths paths, PVNameToKeyMapping pv2key)
static Path
getPathNameForTime(String rootFolder, String pvName, long epochSeconds, boolean useReducedData, PartitionGranularity partitionGranularity, ArchPaths paths, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
static Path
getPathNameForTime(String rootFolder, String pvName, long epochSeconds, PartitionGranularity partitionGranularity, ArchPaths paths, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
static Path[]
getPathsBeforeCurrentPartition(ArchPaths archPaths, String rootFolder, String pvName, Timestamp currentTime, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
The PlainPB storage plugin partitions files according to time and partition granularity.static Path[]
getPathsWithData(ArchPaths archPaths, String rootFolder, String pvName, Timestamp startts, Timestamp endts, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
Given a parent folder, this method returns a list of all the paths with data that falls within the specified timeframe.static Path
getPreviousPartitionBeforeTime(ArchPaths archPaths, String rootFolder, String pvName, Timestamp startts, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
If a PV changes infrequently, we often will not have a sample in the given time frame.static Path
getSparsifiedPathNameForTime(PlainPBStoragePlugin plugin, String pvName, long epochSeconds, ArchPaths paths, PVNameToKeyMapping pv2key)
static Path
getSparsifiedPathNameForTime(String rootFolder, String pvName, long epochSeconds, PartitionGranularity partitionGranularity, ArchPaths paths, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key)
-
-
-
Method Detail
-
getPathNameForTime
public static Path getPathNameForTime(PlainPBStoragePlugin plugin, String pvName, long epochSeconds, ArchPaths paths, PVNameToKeyMapping pv2key) throws IOException
- Throws:
IOException
-
getPathNameForTime
public static Path getPathNameForTime(String rootFolder, String pvName, long epochSeconds, PartitionGranularity partitionGranularity, ArchPaths paths, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws IOException
- Throws:
IOException
-
getSparsifiedPathNameForTime
public static Path getSparsifiedPathNameForTime(PlainPBStoragePlugin plugin, String pvName, long epochSeconds, ArchPaths paths, PVNameToKeyMapping pv2key) throws IOException
- Throws:
IOException
-
getSparsifiedPathNameForTime
public static Path getSparsifiedPathNameForTime(String rootFolder, String pvName, long epochSeconds, PartitionGranularity partitionGranularity, ArchPaths paths, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws IOException
- Throws:
IOException
-
getPathNameForTime
public static Path getPathNameForTime(String rootFolder, String pvName, long epochSeconds, boolean useReducedData, PartitionGranularity partitionGranularity, ArchPaths paths, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws IOException
- Throws:
IOException
-
getPathsWithData
public static Path[] getPathsWithData(ArchPaths archPaths, String rootFolder, String pvName, Timestamp startts, Timestamp endts, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws IOException
Given a parent folder, this method returns a list of all the paths with data that falls within the specified timeframe. We assume the pathnames follow the syntax used by the PlainPBStorage plugin. The alg for matching is based on this-------- [ ] [|] [ ] [|] [ ]
For the chunks that are eliminated, either the end time of the chunk is less than the start time or the start time of the chunk is greater than the end time.- Parameters:
archPaths
- The replacement for NIO PathsrootFolder
- The root folder for the plugin.pvName
- Name of the PV.startts
- Timestamp startendts
- Timestamp endextension
- The file extension.granularity
- Partition granularity of the file.compressionMode
- Compression Modepv2key
- PVNameToKeyMapping- Returns:
- Path A list of all the paths
- Throws:
IOException
-
-
getPathsBeforeCurrentPartition
public static Path[] getPathsBeforeCurrentPartition(ArchPaths archPaths, String rootFolder, String pvName, Timestamp currentTime, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws IOException
The PlainPB storage plugin partitions files according to time and partition granularity. At any particular point in time, we are only writing to one partition, the "current" partition. For ETL, we need to know the partitions that are not being written into; that is, all the previous partitions. This is typically everything except the file for the current partition- Parameters:
archPaths
- The replacement for NIO PathsrootFolder
- The root folder for the pluginpvName
- The name of the PVcurrentTime
- The time that we are running ETL for. To prevent border conditions, caller can add a buffer if needed.extension
- The file extension.granularity
- The granularity of this store.compressionMode
- Compression Modepv2key
- PVNameToKeyMapping- Returns:
- Path A list of all the paths
- Throws:
IOException
-
-
getAllPathsForPV
public static Path[] getAllPathsForPV(ArchPaths archPaths, String rootFolder, String pvName, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws IOException
This method returns all the paths that could contain data for a PV sorted according to the name (which in our case should translate to time).- Parameters:
archPaths
- The replacement for NIO PathsrootFolder
- The root folder for the pluginpvName
- The name of the PVextension
- The file extension.granularity
- The granularity of this store.compressionMode
- Compression Modepv2key
- PVNameToKeyMapping- Returns:
- Path A list of all the paths
- Throws:
IOException
-
-
getMostRecentPathBeforeTime
public static Path getMostRecentPathBeforeTime(ArchPaths archPaths, String rootFolder, String pvName, Timestamp startts, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws Exception
If a PV changes infrequently, we often will not have a sample in the given time frame. The getData contract asks that we return the most recent known sample; even if this sample's timestamp is before the requested start/end time. The way we do this is to ask for the file that potentially has most recent data before the start time. We take advantage of the sorting nature of getAllPathsForPV and work our way from the back- Parameters:
archPaths
- The replacement for NIO PathsrootFolder
- The root folder for the pluginpvName
- Name of the PV.startts
- Timestamp startextension
- The file extension.granularity
- Partition granularity of the file.compressionMode
- Compression Modepv2key
- PVNameToKeyMapping- Returns:
- Path A list of all the paths
- Throws:
Exception
-
-
getPreviousPartitionBeforeTime
public static Path getPreviousPartitionBeforeTime(ArchPaths archPaths, String rootFolder, String pvName, Timestamp startts, String extension, PartitionGranularity granularity, PlainPBStoragePlugin.CompressionMode compressionMode, PVNameToKeyMapping pv2key) throws Exception
If a PV changes infrequently, we often will not have a sample in the given time frame. The getData contract asks that we return the most recent known sample; even if this sample's timestamp is before the requested start/end time. Another way we do this is to return the last event in the partition which ends before the start time. We take advantage of the sorting nature of getAllPathsForPV and work our way from the back- Parameters:
archPaths
- The replacement for NIO PathsrootFolder
- The root folder for the pluginpvName
- Name of the PV.startts
- Timestamp startextension
- The file extension.granularity
- Partition granularity of the file.compressionMode
- Compression Modepv2key
- PVNameToKeyMapping- Returns:
- Path A list of all the paths
- Throws:
Exception
-
-
determineTimesFromFileName
public static edu.stanford.slac.archiverappliance.PlainPB.PlainPBPathNameUtility.StartEndTimeFromName determineTimesFromFileName(String pvName, String finalNameComponent, PartitionGranularity partitionGranularity, PVNameToKeyMapping pv2key) throws IOException
Determines the times for a chunk simply from the file name. Bear in mind there is no guarantee that the file has data in this range. For that, @see PBFileInfo.- Parameters:
pvName
- Name of the PV.finalNameComponent
- The final name componentpartitionGranularity
- Partition granularity of the file.pv2key
- PVNameToKeyMapping- Returns:
- fileNameTimes Start and End Time from name
- Throws:
IOException
-
-
-