Package edu.stanford.slac.archiverappliance.plain.pb

Class PBAppendDataStateData

java.lang.Object

edu.stanford.slac.archiverappliance.plain.AppendDataStateData

edu.stanford.slac.archiverappliance.plain.pb.PBAppendDataStateData

public class PBAppendDataStateData
extends AppendDataStateData

Companion class to PlainStoragePlugin that handles the appending of event streams in a partition-aware fashion. This is used both by the engine and by ETL.

Author:

mshankar

Field Summary

Fields

Modifier and Type	Field	Description
static final int	BULK_BUFFER_INITIAL_CAPACITY

Fields inherited from class edu.stanford.slac.archiverappliance.plain.AppendDataStateData

currentEventsYear, desc, lastKnownTimeStamp, partitionGranularity, previousFilePath, previousYear, pv2key, rootFolder, writer

Constructor Summary

Constructors

Constructor	Description
PBAppendDataStateData(PartitionGranularity partitionGranularity, String rootFolder, String desc, Instant lastKnownTimestamp, PVNameToKeyMapping pv2key, PathResolver pathResolver)

Method Summary

Modifier and Type	Method	Description
boolean	bulkAppend(String pvName, ETLContext context, ETLBulkStream bulkStream, String extension, String extensionToCopyFrom)	Append data in bulk skipping some of the per event checks.
protected EventFileWriter	createNewWriter(String pvName, Path pvPath, EventStream stream)	Append data into PB files honoring partition boundaries switching into new partitions as we cross the boundary.
String	toString()
void	updateStateBasedOnExistingFile(String pvName, Path pvPath)

Methods inherited from class edu.stanford.slac.archiverappliance.plain.AppendDataStateData

checkStream, closeStreams, getPathResolver, partitionBoundaryAwareAppendData, preparePartition, shouldISkipEventBasedOnTimeStamps, shouldISwitchPartitions

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

BULK_BUFFER_INITIAL_CAPACITY

public static final int BULK_BUFFER_INITIAL_CAPACITY

See Also:

• Constant Field Values

Constructor Details

PBAppendDataStateData

public PBAppendDataStateData(PartitionGranularity partitionGranularity,
 String rootFolder,
 String desc,
 Instant lastKnownTimestamp,
 PVNameToKeyMapping pv2key,
 PathResolver pathResolver)

Parameters:

partitionGranularity - partitionGranularity of the PB plugin.

rootFolder - RootFolder of the PB plugin

desc - Desc for logging purposes

lastKnownTimestamp - This is probably the most important argument here. This is the last known timestamp in this storage. If null, we assume time(0) for the last known timestamp.

pv2key - PVNameToKeyMapping

Method Details

createNewWriter

protected EventFileWriter createNewWriter(String pvName,
 Path pvPath,
 EventStream stream)
                                   throws IOException

Append data into PB files honoring partition boundaries switching into new partitions as we cross the boundary.

1. We make sure timestamp monotonicity is maintained.
2. We generate clean partitions.

Specified by:

createNewWriter in class AppendDataStateData

Parameters:

pvName - The PV name

stream -  

Returns:

eventsAppended  

Throws:

IOException -  

updateStateBasedOnExistingFile

public void updateStateBasedOnExistingFile(String pvName,
 Path pvPath)
                                    throws IOException

Specified by:

updateStateBasedOnExistingFile in class AppendDataStateData

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

bulkAppend

public boolean bulkAppend(String pvName,
 ETLContext context,
 ETLBulkStream bulkStream,
 String extension,
 String extensionToCopyFrom)
                   throws IOException

Append data in bulk skipping some of the per event checks.

Specified by:

bulkAppend in class AppendDataStateData

Parameters:

pvName - The PV name

context - The ETL context

bulkStream - The ETL bulk stream

extension -  

extensionToCopyFrom -  

Returns:

boolean  

Throws:

IOException -