Package edu.stanford.slac.archiverappliance.plain.pb

Class PBAppendDataStateData

java.lang.Object

edu.stanford.slac.archiverappliance.plain.AppendDataStateData

edu.stanford.slac.archiverappliance.plain.pb.PBAppendDataStateData

public class PBAppendDataStateData
extends AppendDataStateData

Companion class to PlainStoragePlugin that handles the appending of event streams in a partition-aware fashion. This is used both by the engine and by ETL.

Author:

mshankar

Field Summary

Fields

Modifier and Type	Field	Description
static final int	BULK_BUFFER_INITIAL_CAPACITY

Fields inherited from class edu.stanford.slac.archiverappliance.plain.AppendDataStateData

currentEventsYear, desc, lastKnownTimeStamp, partitionGranularity, previousFilePath, previousYear, pv2key, rootFolder

Constructor Summary

Constructors

Constructor	Description
PBAppendDataStateData(PartitionGranularity partitionGranularity, String rootFolder, String desc, Instant lastKnownTimestamp, PBCompressionMode compressionMode, PVNameToKeyMapping pv2key, PathResolver pathResolver)

Method Summary

Modifier and Type	Method	Description
boolean	bulkAppend(String pvName, ETLContext context, ETLBulkStream bulkStream, String extension, String extensionToCopyFrom)	Append data in bulk skipping some of the per event checks.
void	closeStreams()
protected void	createNewFileAndWriteAHeader(String pvName, Path pvPath, EventStream stream)	In cases where we create a new file, this method is used to create an empty file and write out an header.
int	partitionBoundaryAwareAppendData(BasicContext context, String pvName, EventStream stream, String extension, String extensionToCopyFrom)	Append data into PB files honoring partition boundaries switching into new partitions as we cross the boundary.
String	toString()
void	updateStateBasedOnExistingFile(String pvName, Path pvPath)	If we have an existing file, then this loads a PBInfo, validates the PV name and then updates the appendDataState

Methods inherited from class edu.stanford.slac.archiverappliance.plain.AppendDataStateData

checkStream, getPathResolver, preparePartition, shouldISkipEventBasedOnTimeStamps, shouldISwitchPartitions

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

BULK_BUFFER_INITIAL_CAPACITY

public static final int BULK_BUFFER_INITIAL_CAPACITY

See Also:

• Constant Field Values

Constructor Details

PBAppendDataStateData

public PBAppendDataStateData(PartitionGranularity partitionGranularity,
 String rootFolder,
 String desc,
 Instant lastKnownTimestamp,
 PBCompressionMode compressionMode,
 PVNameToKeyMapping pv2key,
 PathResolver pathResolver)

Parameters:

partitionGranularity - partitionGranularity of the PB plugin.

rootFolder - RootFolder of the PB plugin

desc - Desc for logging purposes

lastKnownTimestamp - This is probably the most important argument here. This is the last known timestamp in this storage. If null, we assume time(0) for the last known timestamp.

compressionMode - CompressionMode

pv2key - PVNameToKeyMapping

Method Details

partitionBoundaryAwareAppendData

public int partitionBoundaryAwareAppendData(BasicContext context,
 String pvName,
 EventStream stream,
 String extension,
 String extensionToCopyFrom)
                                     throws IOException

Append data into PB files honoring partition boundaries switching into new partitions as we cross the boundary.

1. We make sure timestamp monotonicity is maintained.
2. We generate clean partitions.

Specified by:

partitionBoundaryAwareAppendData in class AppendDataStateData

Parameters:

context -  

pvName - The PV name

stream -  

extension -  

extensionToCopyFrom -  

Returns:

eventsAppended  

Throws:

IOException -  

closeStreams

public void closeStreams()

Specified by:

closeStreams in class AppendDataStateData

updateStateBasedOnExistingFile

public void updateStateBasedOnExistingFile(String pvName,
 Path pvPath)
                                    throws IOException

If we have an existing file, then this loads a PBInfo, validates the PV name and then updates the appendDataState

Specified by:

updateStateBasedOnExistingFile in class AppendDataStateData

Parameters:

pvName - The PV name

pvPath - The PV path

Throws:

IOException -  

createNewFileAndWriteAHeader

protected void createNewFileAndWriteAHeader(String pvName,
 Path pvPath,
 EventStream stream)
                                     throws IOException

In cases where we create a new file, this method is used to create an empty file and write out an header.

Specified by:

createNewFileAndWriteAHeader in class AppendDataStateData

Parameters:

pvName - The PV name

pvPath - The PV path

stream - The Event stream

Throws:

IOException -  

toString

public String toString()

Overrides:

toString in class Object

bulkAppend

public boolean bulkAppend(String pvName,
 ETLContext context,
 ETLBulkStream bulkStream,
 String extension,
 String extensionToCopyFrom)
                   throws IOException

Append data in bulk skipping some of the per event checks.

Specified by:

bulkAppend in class AppendDataStateData

Parameters:

pvName - The PV name

context - The ETL context

bulkStream - The ETL bulk stream

extension -  

extensionToCopyFrom -  

Returns:

boolean  

Throws:

IOException -