edu.iu.dsc.tws.api.tset.sets.batch

Interface BatchTSet<T>

Type Parameters:

T - tset type

All Superinterfaces:

AcceptingData<T>, java.io.Serializable, StoringData<T>, TBase, TSet<T>

All Known Implementing Classes:

BatchTSetImpl, CachedTSet, CheckpointedTSet, ComputeTSet, PersistedTSet, SourceTSet, StoredTSet

public interface BatchTSet<T>
extends TSet<T>, AcceptingData<T>, StoringData<T>

Batch extension of TSet interface. Adds methods related to Batch OperationMode and overrides generic TSet and TLink return type to BatchTSet and BatchTLink in all methods. Also extends the AcceptingData interface because the BatchTSets can accept data from other TSets.

Method Summary

Modifier and Type	Method and Description
BatchTSet<T>	addInput(java.lang.String key, StorableTBase<?> input) Adds data to this TSet
BatchTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T>	allGather() Same as gather, but all the target TSet instances would receive the gathered result in the runtime.
BatchTLink<T,T>	allReduce(ReduceFunc<T> reduceFn) Similar to reduce, but all instances of the target TSet would receive the reduced result.
BatchTLink<java.util.Iterator<T>,T>	direct() Returns a Direct TLink that corresponds to the communication operation where the data will be transferred to another TSet directly.
BatchTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T>	gather() Returns a Gather TLink that would gather data to the target TSet instance with index 0 (in the runtime).
<K,V> BatchTupleTSet<K,V>	mapToTuple(MapFunc<Tuple<K,V>,T> mapToTupleFn) Creates a TupleTSet based on the MapFunc provided.
BatchTLink<java.util.Iterator<T>,T>	partition(PartitionFunc<T> partitionFn) Same as above, but the parallelism will be preserved in the target TSet.
BatchTLink<java.util.Iterator<T>,T>	partition(PartitionFunc<T> partitionFn, int targetParallelism) Returns a Partition TLink that would partition data according based on a function provided.
BatchTLink<T,T>	pipe()
BatchTLink<T,T>	reduce(ReduceFunc<T> reduceFn) Returns a Reduce TLink that reduce data on to the target TSet instance (in the runtime) with index 0.
BatchTLink<java.util.Iterator<T>,T>	replicate(int replications) Returns a Replicate TLink that would clone/broadcast the data from this TSet.
BatchTSet<T>	setName(java.lang.String name) Sets the name
BatchTSet<T>	union(java.util.Collection<TSet<T>> tSets) Same as above, but accepts a Collection of TSets.
BatchTSet<T>	union(TSet<T> unionTSet) Returns a single TSet that create a union of data in both TSets.

Methods inherited from interface edu.iu.dsc.tws.api.tset.sets.TSet

withSchema

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName

Methods inherited from interface edu.iu.dsc.tws.api.tset.StoringData

cache, lazyCache, lazyPersist, persist

Method Detail

setName

BatchTSet<T> setName(java.lang.String name)

Description copied from interface: TSet

Sets the name

Specified by:

setName in interface TBase

Specified by:

setName in interface TSet<T>

direct

BatchTLink<java.util.Iterator<T>,T> direct()

Description copied from interface: TSet

Returns a Direct TLink that corresponds to the communication operation where the data will be transferred to another TSet directly.

Specified by:

direct in interface TSet<T>

Returns:

Direct TLink

pipe

BatchTLink<T,T> pipe()

reduce

BatchTLink<T,T> reduce(ReduceFunc<T> reduceFn)

Description copied from interface: TSet

Returns a Reduce TLink that reduce data on to the target TSet instance (in the runtime) with index 0.

Specified by:

reduce in interface TSet<T>

Parameters:

reduceFn - Reduce function

Returns:

Reduce TLink

allReduce

BatchTLink<T,T> allReduce(ReduceFunc<T> reduceFn)

Description copied from interface: TSet

Similar to reduce, but all instances of the target TSet would receive the reduced result.

Specified by:

allReduce in interface TSet<T>

Parameters:

reduceFn - Reduce function

Returns:

AllReduce TLink

partition

BatchTLink<java.util.Iterator<T>,T> partition(PartitionFunc<T> partitionFn,
                                              int targetParallelism)

Description copied from interface: TSet

Returns a Partition TLink that would partition data according based on a function provided. The parallelism of the target TSet can also be specified.

Specified by:

partition in interface TSet<T>

Parameters:

partitionFn - Partition function

targetParallelism - Target parallelism

Returns:

Partition TLink

partition

BatchTLink<java.util.Iterator<T>,T> partition(PartitionFunc<T> partitionFn)

Description copied from interface: TSet

Same as above, but the parallelism will be preserved in the target TSet.

Specified by:

partition in interface TSet<T>

Parameters:

partitionFn - Partition function

Returns:

Partition TLink

gather

BatchTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T> gather()

Description copied from interface: TSet

Returns a Gather TLink that would gather data to the target TSet instance with index 0 (in the runtime).

Specified by:

gather in interface TSet<T>

Returns:

Gather TLink

allGather

BatchTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T> allGather()

Description copied from interface: TSet

Same as gather, but all the target TSet instances would receive the gathered result in the runtime.

Specified by:

allGather in interface TSet<T>

Returns:

AllGather TLink

mapToTuple

<K,V> BatchTupleTSet<K,V> mapToTuple(MapFunc<Tuple<K,V>,T> mapToTupleFn)

Description copied from interface: TSet

Creates a TupleTSet based on the MapFunc provided. This will an entry point to keyed communication operations from a non-keyed TSet.

Specified by:

mapToTuple in interface TSet<T>

Type Parameters:

K - type of key

V - type of value

Parameters:

mapToTupleFn - Map function

Returns:

Tuple TSet

replicate

BatchTLink<java.util.Iterator<T>,T> replicate(int replications)

Description copied from interface: TSet

Returns a Replicate TLink that would clone/broadcast the data from this TSet. Note that the parallelism of this TSet should be 1.

Specified by:

replicate in interface TSet<T>

Parameters:

replications - Replicas of data (= target TSet parallelism)

Returns:

Replicate TLink

union

BatchTSet<T> union(TSet<T> unionTSet)

Description copied from interface: TSet

Returns a single TSet that create a union of data in both TSets. In order for this to work both TSets should be of the same type

Specified by:

union in interface TSet<T>

Parameters:

unionTSet - TSet to union with

Returns:

Union TSet

union

BatchTSet<T> union(java.util.Collection<TSet<T>> tSets)

Description copied from interface: TSet

Same as above, but accepts a Collection of TSets.

Specified by:

union in interface TSet<T>

Parameters:

tSets - a collection of TSet's to union with

Returns:

Union TSet

addInput

BatchTSet<T> addInput(java.lang.String key,
                      StorableTBase<?> input)

Adds data to this TSet

Specified by:

addInput in interface AcceptingData<T>

Parameters:

key - the key used to store the given TSet

input - a @StorableTBase TSet to be added as an input

Returns:

this test