edu.iu.dsc.tws.api.tset.sets.streaming

Interface StreamingTSet<T>

Type Parameters:

T - tset type

All Superinterfaces:

java.io.Serializable, TBase, TSet<T>

All Known Implementing Classes:

SComputeTSet, SSinkTSet, SSourceTSet, StreamingTSetImpl, WindowComputeTSet

public interface StreamingTSet<T>
extends TSet<T>

Batch extension of TSet interface. Adds methods related to Batch OperationMode and overrides generic TSet and TLink return type to StreamingTSet and StreamingTLink in all methods.

Method Summary

Modifier and Type	Method and Description
StreamingTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T>	allGather() Same as gather, but all the target TSet instances would receive the gathered result in the runtime.
StreamingTLink<T,T>	allReduce(ReduceFunc<T> reduceFn) Similar to reduce, but all instances of the target TSet would receive the reduced result.
StreamingTLink<T,T>	direct() Returns a Direct TLink that corresponds to the communication operation where the data will be transferred to another TSet directly.
StreamingTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T>	gather() Returns a Gather TLink that would gather data to the target TSet instance with index 0 (in the runtime).
<K,V> StreamingTupleTSet<K,V>	mapToTuple(MapFunc<Tuple<K,V>,T> mapToTupleFn) Creates a TupleTSet based on the MapFunc provided.
StreamingTLink<T,T>	partition(PartitionFunc<T> partitionFn) Same as above, but the parallelism will be preserved in the target TSet.
StreamingTLink<T,T>	partition(PartitionFunc<T> partitionFn, int targetParallelism) Returns a Partition TLink that would partition data according based on a function provided.
StreamingTLink<T,T>	reduce(ReduceFunc<T> reduceFn) Returns a Reduce TLink that reduce data on to the target TSet instance (in the runtime) with index 0.
StreamingTLink<T,T>	replicate(int replications) Returns a Replicate TLink that would clone/broadcast the data from this TSet.
StreamingTSet<T>	setName(java.lang.String name) Sets the name
StreamingTSet<T>	union(java.util.Collection<TSet<T>> tSets) Same as above, but accepts a Collection of TSets.
StreamingTSet<T>	union(TSet<T> unionTSet) Returns a single TSet that create a union of data in both TSets.

Methods inherited from interface edu.iu.dsc.tws.api.tset.sets.TSet

withSchema

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName

Method Detail

setName

StreamingTSet<T> setName(java.lang.String name)

Description copied from interface: TSet

Sets the name

Specified by:

setName in interface TBase

Specified by:

setName in interface TSet<T>

direct

StreamingTLink<T,T> direct()

Description copied from interface: TSet

Returns a Direct TLink that corresponds to the communication operation where the data will be transferred to another TSet directly.

Specified by:

direct in interface TSet<T>

Returns:

Direct TLink

reduce

StreamingTLink<T,T> reduce(ReduceFunc<T> reduceFn)

Description copied from interface: TSet

Returns a Reduce TLink that reduce data on to the target TSet instance (in the runtime) with index 0.

Specified by:

reduce in interface TSet<T>

Parameters:

reduceFn - Reduce function

Returns:

Reduce TLink

allReduce

StreamingTLink<T,T> allReduce(ReduceFunc<T> reduceFn)

Description copied from interface: TSet

Similar to reduce, but all instances of the target TSet would receive the reduced result.

Specified by:

allReduce in interface TSet<T>

Parameters:

reduceFn - Reduce function

Returns:

AllReduce TLink

partition

StreamingTLink<T,T> partition(PartitionFunc<T> partitionFn,
                              int targetParallelism)

Description copied from interface: TSet

Returns a Partition TLink that would partition data according based on a function provided. The parallelism of the target TSet can also be specified.

Specified by:

partition in interface TSet<T>

Parameters:

partitionFn - Partition function

targetParallelism - Target parallelism

Returns:

Partition TLink

partition

StreamingTLink<T,T> partition(PartitionFunc<T> partitionFn)

Description copied from interface: TSet

Same as above, but the parallelism will be preserved in the target TSet.

Specified by:

partition in interface TSet<T>

Parameters:

partitionFn - Partition function

Returns:

Partition TLink

gather

StreamingTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T> gather()

Description copied from interface: TSet

Returns a Gather TLink that would gather data to the target TSet instance with index 0 (in the runtime).

Specified by:

gather in interface TSet<T>

Returns:

Gather TLink

allGather

StreamingTLink<java.util.Iterator<Tuple<java.lang.Integer,T>>,T> allGather()

Description copied from interface: TSet

Same as gather, but all the target TSet instances would receive the gathered result in the runtime.

Specified by:

allGather in interface TSet<T>

Returns:

AllGather TLink

mapToTuple

<K,V> StreamingTupleTSet<K,V> mapToTuple(MapFunc<Tuple<K,V>,T> mapToTupleFn)

Description copied from interface: TSet

Creates a TupleTSet based on the MapFunc provided. This will an entry point to keyed communication operations from a non-keyed TSet.

Specified by:

mapToTuple in interface TSet<T>

Type Parameters:

K - type of key

V - type of value

Parameters:

mapToTupleFn - Map function

Returns:

Tuple TSet

replicate

StreamingTLink<T,T> replicate(int replications)

Description copied from interface: TSet

Returns a Replicate TLink that would clone/broadcast the data from this TSet. Note that the parallelism of this TSet should be 1.

Specified by:

replicate in interface TSet<T>

Parameters:

replications - Replicas of data (= target TSet parallelism)

Returns:

Replicate TLink

union

StreamingTSet<T> union(TSet<T> unionTSet)

Description copied from interface: TSet

Returns a single TSet that create a union of data in both TSets. In order for this to work both TSets should be of the same type

Specified by:

union in interface TSet<T>

Parameters:

unionTSet - TSet to union with

Returns:

Union TSet

union

StreamingTSet<T> union(java.util.Collection<TSet<T>> tSets)

Description copied from interface: TSet

Same as above, but accepts a Collection of TSets.

Specified by:

union in interface TSet<T>

Parameters:

tSets - a collection of TSet's to union with

Returns:

Union TSet