T - base type of the tsetpublic abstract class StoredTSet<T> extends BatchTSetImpl<T> implements StorableTBase<T>
BaseTSet.StateType| Modifier and Type | Field and Description |
|---|---|
protected SourceTSet<T> |
storedSource |
| Modifier and Type | Method and Description |
|---|---|
StoredTSet<T> |
addInput(java.lang.String key,
StorableTBase<?> input)
Adds data to this TSet
|
AllGatherTLink<T> |
allGather()
Same as gather, but all the target TSet instances would receive the gathered result in the
runtime.
|
AllReduceTLink<T> |
allReduce(ReduceFunc<T> reduceFn)
Similar to reduce, but all instances of the target
TSet would receive the reduced
result. |
DirectTLink<T> |
direct()
Returns a Direct
TLink that corresponds to the communication operation where the data
will be transferred to another TSet directly. |
GatherTLink<T> |
gather()
Returns a Gather
TLink that would gather data to the target TSet instance with index
0 (in the runtime). |
java.util.List<T> |
getData()
retrieve data saved in the TSet as a
List |
DataObject<T> |
getDataObject() |
INode |
getINode() |
SourceTSet<T> |
getStoredSourceTSet()
Once a cache TSet is run, it will produce data and store data in the data executor with the
tset ID as the key.
|
<K,V> KeyedTSet<K,V> |
mapToTuple(MapFunc<Tuple<K,V>,T> generateTuple)
|
PartitionTLink<T> |
partition(PartitionFunc<T> partitionFn)
Same as above, but the parallelism will be preserved in the target
TSet. |
PartitionTLink<T> |
partition(PartitionFunc<T> partitionFn,
int targetParallelism)
Returns a Partition
TLink that would partition data according based on a function
provided. |
PipeTLink<T> |
pipe() |
ReduceTLink<T> |
reduce(ReduceFunc<T> reduceFn)
Returns a Reduce
TLink that reduce data on to the target TSet instance (in the runtime)
with index 0. |
ReplicateTLink<T> |
replicate(int replications)
|
ComputeTSet<T,java.util.Iterator<T>> |
union(java.util.Collection<TSet<T>> tSets)
Same as above, but accepts a
Collection of TSets. |
ComputeTSet<T,java.util.Iterator<T>> |
union(TSet<T> other)
|
cache, getTSetEnv, lazyCache, lazyPersist, persist, withSchemagetInputSchema, getOutputSchema, setOutputSchemaaddChildToGraph, addChildToGraph, equals, getId, getInputs, getName, getParallelism, getStateType, hashCode, isMutable, rename, setMutable, setStateType, setTSetEnv, toStringclone, finalize, getClass, notify, notifyAll, wait, wait, waitbuildgenerateID, getTBaseGraphprotected SourceTSet<T> storedSource
public INode getINode()
getINode in interface BuildableTSetpublic DirectTLink<T> direct()
TSetTLink that corresponds to the communication operation where the data
will be transferred to another TSet directly.public ReduceTLink<T> reduce(ReduceFunc<T> reduceFn)
TSetTLink that reduce data on to the target TSet instance (in the runtime)
with index 0.public PartitionTLink<T> partition(PartitionFunc<T> partitionFn, int targetParallelism)
TSetpublic PartitionTLink<T> partition(PartitionFunc<T> partitionFn)
TSetTSet.public GatherTLink<T> gather()
TSetTLink that would gather data to the target TSet instance with index
0 (in the runtime).public AllReduceTLink<T> allReduce(ReduceFunc<T> reduceFn)
TSetTSet would receive the reduced
result.public AllGatherTLink<T> allGather()
TSetpublic <K,V> KeyedTSet<K,V> mapToTuple(MapFunc<Tuple<K,V>,T> generateTuple)
TSetTupleTSet based on the MapFunc provided. This will an entry point
to keyed communication operations from a non-keyed TSet.mapToTuple in interface BatchTSet<T>mapToTuple in interface TSet<T>mapToTuple in class BatchTSetImpl<T>K - type of keyV - type of valuegenerateTuple - Map functionpublic ReplicateTLink<T> replicate(int replications)
TSetpublic StoredTSet<T> addInput(java.lang.String key, StorableTBase<?> input)
BatchTSetaddInput in interface AcceptingData<T>addInput in interface BatchTSet<T>addInput in class BatchTSetImpl<T>key - the key used to store the given TSetinput - a @StorableTBase TSet to be added as an inputpublic ComputeTSet<T,java.util.Iterator<T>> union(TSet<T> other)
TSetpublic ComputeTSet<T,java.util.Iterator<T>> union(java.util.Collection<TSet<T>> tSets)
TSetCollection of TSets.public SourceTSet<T> getStoredSourceTSet()
SourceTSetgetStoredSourceTSet in interface StorableTBase<T>SourceTSet source tset from data
public java.util.List<T> getData()
StorableTBaseList
NOTE: use this method only when you need to pull the data from the tset. Otherwise this would unnecessarily loads data to the memory.
getData in interface StorableTBase<T>public DataObject<T> getDataObject()
getDataObject in interface StorableTBase<T>