edu.iu.dsc.tws.tset.sets.batch

Class KeyedCheckpointedTSet<K,V>

java.lang.Object

edu.iu.dsc.tws.tset.sets.BaseTSet<T>

edu.iu.dsc.tws.tset.sets.BaseTSetWithSchema<V>

edu.iu.dsc.tws.tset.sets.batch.BatchTupleTSetImpl<K,V>

edu.iu.dsc.tws.tset.sets.batch.KeyedStoredTSet<K,V>

edu.iu.dsc.tws.tset.sets.batch.KeyedPersistedTSet<K,V>

edu.iu.dsc.tws.tset.sets.batch.KeyedCheckpointedTSet<K,V>

Type Parameters:

K - tset key type

All Implemented Interfaces:

AcceptingData<Tuple<K,V>>, BatchTupleTSet<K,V>, StorableTBase<Tuple<K,V>>, TupleTSet<K,V>, StoringData<Tuple<K,V>>, TBase, Buildable, BuildableTSet, java.io.Serializable

public class KeyedCheckpointedTSet<K,V>
extends KeyedPersistedTSet<K,V>

This is a shadow KeyedPersistedTSet to add the checkpointing capability. It does not have the sink that would store the data, because the purpose of this tset is to expose the data that was stored by a KeyedPersistedTSet.

When this tset is executed, it would wrap DiskPartitionBackedSource from DiskPartitionBackedSourceWrapper and return a CheckpointedSourceOp as the INode for the underlying task.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class edu.iu.dsc.tws.tset.sets.BaseTSet

BaseTSet.StateType

Field Summary

Fields inherited from class edu.iu.dsc.tws.tset.sets.batch.KeyedStoredTSet

storedSource

Constructor Summary

Constructors

Constructor and Description
KeyedCheckpointedTSet(BatchEnvironment tSetEnv, DiskPartitionBackedSource<Tuple<K,V>> sourceFn, int parallelism, KeyedSchema inputSchema)

Method Summary

Modifier and Type	Method and Description
INode	getINode() Reuses the DiskPartitionBackedSourceWrapper from non-keyed operations because there will be no keyed writes to edges.
KeyedSourceTSet<K,V>	getStoredSourceTSet() retrieve data saved in the TSet

Methods inherited from class edu.iu.dsc.tws.tset.sets.batch.KeyedPersistedTSet

addInput, cache, lazyCache, lazyPersist, persist, setName, withSchema

Methods inherited from class edu.iu.dsc.tws.tset.sets.batch.KeyedStoredTSet

getData, getInputSchema, join, join, keyedDirect, keyedGather, keyedGather, keyedGather, keyedGatherUngrouped, keyedGatherUngrouped, keyedGatherUngrouped, keyedPartition, keyedPipe, keyedReduce

Methods inherited from class edu.iu.dsc.tws.tset.sets.batch.BatchTupleTSetImpl

getOutputSchema, getTSetEnv

Methods inherited from class edu.iu.dsc.tws.tset.sets.BaseTSetWithSchema

setOutputSchema

Methods inherited from class edu.iu.dsc.tws.tset.sets.BaseTSet

addChildToGraph, addChildToGraph, equals, getId, getInputs, getName, getParallelism, getStateType, hashCode, isMutable, rename, setMutable, setStateType, setTSetEnv, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.tset.sets.StorableTBase

getDataObject

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName

Methods inherited from interface edu.iu.dsc.tws.tset.sets.BuildableTSet

build

Methods inherited from interface edu.iu.dsc.tws.tset.Buildable

generateID, getTBaseGraph

Constructor Detail

KeyedCheckpointedTSet

public KeyedCheckpointedTSet(BatchEnvironment tSetEnv,
                             DiskPartitionBackedSource<Tuple<K,V>> sourceFn,
                             int parallelism,
                             KeyedSchema inputSchema)

Method Detail

getStoredSourceTSet

public KeyedSourceTSet<K,V> getStoredSourceTSet()

Description copied from interface: StorableTBase

retrieve data saved in the TSet

Specified by:

getStoredSourceTSet in interface StorableTBase<Tuple<K,V>>

Overrides:

getStoredSourceTSet in class KeyedStoredTSet<K,V>

Returns:

dataObject

getINode

public INode getINode()

Reuses the DiskPartitionBackedSourceWrapper from non-keyed operations because there will be no keyed writes to edges. This will only expose the DataPartition that will be called by the TaskExecutor

Specified by:

getINode in interface BuildableTSet

Overrides:

getINode in class KeyedStoredTSet<K,V>

Returns:

INode