edu.iu.dsc.tws.api.tset.sets.batch

Interface BatchTupleTSet<K,V>

Type Parameters:

K -

V -

All Superinterfaces:

AcceptingData<Tuple<K,V>>, java.io.Serializable, StoringData<Tuple<K,V>>, TBase, TupleTSet<K,V>

All Known Implementing Classes:

BatchTupleTSetImpl, KeyedCachedTSet, KeyedCheckpointedTSet, KeyedPersistedTSet, KeyedSourceTSet, KeyedStoredTSet, KeyedTSet

public interface BatchTupleTSet<K,V>
extends TupleTSet<K,V>, AcceptingData<Tuple<K,V>>, StoringData<Tuple<K,V>>

Batch extension of TupleTSet interface with AcceptingData and StoringData capabilities.

Method Summary

Modifier and Type	Method and Description
BatchTupleTSet<K,V>	addInput(java.lang.String key, StorableTBase<?> input) Adds inputs to BatchTupleTSets
<VR> BatchTLink<java.util.Iterator<JoinedTuple<K,V,VR>>,JoinedTuple<K,V,VR>>	join(BatchTupleTSet<K,VR> rightTSet, CommunicationContext.JoinType type, java.util.Comparator<K> keyComparator) Joins with another BatchTupleTSet.
<VR> BatchTLink<java.util.Iterator<JoinedTuple<K,V,VR>>,JoinedTuple<K,V,VR>>	join(BatchTupleTSet<K,VR> rightTSet, CommunicationContext.JoinType type, java.util.Comparator<K> keyComparator, TaskPartitioner<K> partitioner) Joins with another BatchTupleTSet.
BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>>	keyedDirect() Direct/pipe communication
BatchTLink<java.util.Iterator<Tuple<K,java.util.Iterator<V>>>,Tuple<K,java.util.Iterator<V>>>	keyedGather() Gathers data by key for BatchTupleTSets
BatchTLink<java.util.Iterator<Tuple<K,java.util.Iterator<V>>>,Tuple<K,java.util.Iterator<V>>>	keyedGather(PartitionFunc<K> partitionFn) Gathers data by key for BatchTupleTSets
BatchTLink<java.util.Iterator<Tuple<K,java.util.Iterator<V>>>,Tuple<K,java.util.Iterator<V>>>	keyedGather(PartitionFunc<K> partitionFn, java.util.Comparator<K> comparator) Gathers data by key for BatchTupleTSets
BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>>	keyedGatherUngrouped() Gathers data by key for BatchTupleTSets without grouping data by keys
BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>>	keyedGatherUngrouped(PartitionFunc<K> partitionFn) Gathers data by key for BatchTupleTSets without grouping data by keys
BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>>	keyedGatherUngrouped(PartitionFunc<K> partitionFn, java.util.Comparator<K> comparator) Gathers data by key for BatchTupleTSets without grouping data by keys
BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>>	keyedPartition(PartitionFunc<K> partitionFn) Partitions data using a PartitionFunc based on keys
BatchTLink<Tuple<K,V>,Tuple<K,V>>	keyedPipe() Pipe implementation
BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>>	keyedReduce(ReduceFunc<V> reduceFn) Reduces data by key for BatchTupleTSets
BatchTupleTSet<K,V>	setName(java.lang.String name) Name of the tset

Methods inherited from interface edu.iu.dsc.tws.api.tset.sets.TupleTSet

withSchema

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName

Methods inherited from interface edu.iu.dsc.tws.api.tset.StoringData

cache, lazyCache, lazyPersist, persist

Method Detail

setName

BatchTupleTSet<K,V> setName(java.lang.String name)

Name of the tset

Specified by:

setName in interface TBase

Specified by:

setName in interface TupleTSet<K,V>

Parameters:

name - name

Returns:

this set

keyedPartition

BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>> keyedPartition(PartitionFunc<K> partitionFn)

Partitions data using a PartitionFunc based on keys

Specified by:

keyedPartition in interface TupleTSet<K,V>

Parameters:

partitionFn - partition function

Returns:

Keyed Partition TLink

keyedDirect

BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>> keyedDirect()

Direct/pipe communication

Specified by:

keyedDirect in interface TupleTSet<K,V>

Returns:

Keyed Direct TLink

keyedPipe

BatchTLink<Tuple<K,V>,Tuple<K,V>> keyedPipe()

Pipe implementation

Returns:

Keyed Pipe TLink

keyedGather

BatchTLink<java.util.Iterator<Tuple<K,java.util.Iterator<V>>>,Tuple<K,java.util.Iterator<V>>> keyedGather()

Gathers data by key for BatchTupleTSets

Returns:

Keyed Gather TLink

keyedGather

BatchTLink<java.util.Iterator<Tuple<K,java.util.Iterator<V>>>,Tuple<K,java.util.Iterator<V>>> keyedGather(PartitionFunc<K> partitionFn)

Gathers data by key for BatchTupleTSets

Parameters:

partitionFn - partition function to partition data based on key

Returns:

Keyed Gather TLink

keyedGather

BatchTLink<java.util.Iterator<Tuple<K,java.util.Iterator<V>>>,Tuple<K,java.util.Iterator<V>>> keyedGather(PartitionFunc<K> partitionFn,
                                                                                                          java.util.Comparator<K> comparator)

Gathers data by key for BatchTupleTSets

Parameters:

partitionFn - partition function to partition data based on key

comparator - custom key comparator

Returns:

Keyed Gather TLink

keyedGatherUngrouped

BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>> keyedGatherUngrouped()

Gathers data by key for BatchTupleTSets without grouping data by keys

Returns:

this TSet

keyedGatherUngrouped

BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>> keyedGatherUngrouped(PartitionFunc<K> partitionFn)

Gathers data by key for BatchTupleTSets without grouping data by keys

Parameters:

partitionFn - partition function to partition data based on key

Returns:

Keyed Gather TLink

keyedGatherUngrouped

BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>> keyedGatherUngrouped(PartitionFunc<K> partitionFn,
                                                                           java.util.Comparator<K> comparator)

Gathers data by key for BatchTupleTSets without grouping data by keys

Parameters:

partitionFn - partition function to partition data based on key

comparator - custom key comparator

Returns:

Keyed Gather TLink

keyedReduce

BatchTLink<java.util.Iterator<Tuple<K,V>>,Tuple<K,V>> keyedReduce(ReduceFunc<V> reduceFn)

Reduces data by key for BatchTupleTSets

Parameters:

reduceFn - the reduce function

Returns:

Keyed Reduce TLink

join

<VR> BatchTLink<java.util.Iterator<JoinedTuple<K,V,VR>>,JoinedTuple<K,V,VR>> join(BatchTupleTSet<K,VR> rightTSet,
                                                                                  CommunicationContext.JoinType type,
                                                                                  java.util.Comparator<K> keyComparator)

Joins with another BatchTupleTSet. Note that this TSet will be considered the left TSet

Type Parameters:

VR - value type of the right tset

Parameters:

rightTSet - right tset

type - CommunicationContext.JoinType

keyComparator - key comparator

Returns:

Joined TLink

join

<VR> BatchTLink<java.util.Iterator<JoinedTuple<K,V,VR>>,JoinedTuple<K,V,VR>> join(BatchTupleTSet<K,VR> rightTSet,
                                                                                  CommunicationContext.JoinType type,
                                                                                  java.util.Comparator<K> keyComparator,
                                                                                  TaskPartitioner<K> partitioner)

Joins with another BatchTupleTSet. Note that this TSet will be considered the left TSet

Type Parameters:

VR - value type of the right tset

Parameters:

rightTSet - right tset

type - CommunicationContext.JoinType

keyComparator - key comparator

partitioner - partitioner for keys

Returns:

Joined TLink

addInput

BatchTupleTSet<K,V> addInput(java.lang.String key,
                             StorableTBase<?> input)

Adds inputs to BatchTupleTSets

Specified by:

addInput in interface AcceptingData<Tuple<K,V>>

Parameters:

key - identifier for the input

input - input tset

Returns:

this tset