T - tset typepublic class CheckpointedTSet<T> extends PersistedTSet<T>
PersistedTSet to add the checkpointing capability. It does not have
the sink that would store the data, because the purpose of this tset is to expose the data
that was stored by a PersistedTSet.
When this tset is executed, it would wrap DiskPartitionBackedSource from
DiskPartitionBackedSourceWrapper and return a CheckpointedSourceOp as the
INode for the underlying task.
BaseTSet.StateTypestoredSource| Constructor and Description |
|---|
CheckpointedTSet(BatchEnvironment tSetEnv,
DiskPartitionBackedSource<T> sourceFn,
int parallelism,
Schema inputSchema) |
| Modifier and Type | Method and Description |
|---|---|
INode |
getINode() |
SourceTSet<T> |
getStoredSourceTSet()
Once a cache TSet is run, it will produce data and store data in the data executor with the
tset ID as the key.
|
CheckpointedTSet<T> |
withSchema(Schema schema)
Sets the data type of the
TSet output. |
addInput, cache, lazyCache, lazyPersist, persist, setNameallGather, allReduce, direct, gather, getData, getDataObject, mapToTuple, partition, partition, pipe, reduce, replicate, union, uniongetTSetEnvgetInputSchema, getOutputSchema, setOutputSchemaaddChildToGraph, addChildToGraph, equals, getId, getInputs, getName, getParallelism, getStateType, hashCode, isMutable, rename, setMutable, setStateType, setTSetEnv, toStringclone, finalize, getClass, notify, notifyAll, wait, wait, waitbuildgenerateID, getTBaseGraphpublic CheckpointedTSet(BatchEnvironment tSetEnv, DiskPartitionBackedSource<T> sourceFn, int parallelism, Schema inputSchema)
public SourceTSet<T> getStoredSourceTSet()
StoredTSetSourceTSetgetStoredSourceTSet in interface StorableTBase<T>getStoredSourceTSet in class StoredTSet<T>SourceTSet source tset from data
public CheckpointedTSet<T> withSchema(Schema schema)
TSetTSet output. This will be used in the packers for efficient
SER-DE operations in the following TLinkswithSchema in interface TSet<T>withSchema in class PersistedTSet<T>schema - data type as a MessageTypeTSetpublic INode getINode()
getINode in interface BuildableTSetgetINode in class StoredTSet<T>