edu.iu.dsc.tws.tset.sets.batch

Class KeyedStoredTSet<K,V>

java.lang.Object

edu.iu.dsc.tws.tset.sets.BaseTSet<T>

edu.iu.dsc.tws.tset.sets.BaseTSetWithSchema<V>

edu.iu.dsc.tws.tset.sets.batch.BatchTupleTSetImpl<K,V>

edu.iu.dsc.tws.tset.sets.batch.KeyedStoredTSet<K,V>

All Implemented Interfaces:

AcceptingData<Tuple<K,V>>, BatchTupleTSet<K,V>, StorableTBase<Tuple<K,V>>, TupleTSet<K,V>, StoringData<Tuple<K,V>>, TBase, Buildable, BuildableTSet, java.io.Serializable

Direct Known Subclasses:

KeyedCachedTSet, KeyedPersistedTSet

public abstract class KeyedStoredTSet<K,V>
extends BatchTupleTSetImpl<K,V>
implements StorableTBase<Tuple<K,V>>

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class edu.iu.dsc.tws.tset.sets.BaseTSet

BaseTSet.StateType

Field Summary

Fields

Modifier and Type	Field and Description
protected KeyedSourceTSet<K,V>	storedSource

Method Summary

Modifier and Type	Method and Description
java.util.List<Tuple<K,V>>	getData() retrieve data saved in the TSet as a List
INode	getINode()
KeyedSchema	getInputSchema()
KeyedSourceTSet<K,V>	getStoredSourceTSet() retrieve data saved in the TSet
<VR> JoinTLink<K,V,VR>	join(BatchTupleTSet<K,VR> rightTSet, CommunicationContext.JoinType type, java.util.Comparator<K> keyComparator) Joins with another BatchTupleTSet.
<VR> JoinTLink<K,V,VR>	join(BatchTupleTSet<K,VR> rightTSet, CommunicationContext.JoinType type, java.util.Comparator<K> keyComparator, TaskPartitioner<K> partitioner) Joins with another BatchTupleTSet.
KeyedDirectTLink<K,V>	keyedDirect() Direct/pipe communication
KeyedGatherTLink<K,V>	keyedGather() Gathers data by key for BatchTupleTSets
KeyedGatherTLink<K,V>	keyedGather(PartitionFunc<K> partitionFn) Gathers data by key for BatchTupleTSets
KeyedGatherTLink<K,V>	keyedGather(PartitionFunc<K> partitionFn, java.util.Comparator<K> comparator) Gathers data by key for BatchTupleTSets
KeyedGatherUngroupedTLink<K,V>	keyedGatherUngrouped() Gathers data by key for BatchTupleTSets without grouping data by keys
KeyedGatherUngroupedTLink<K,V>	keyedGatherUngrouped(PartitionFunc<K> partitionFn) Gathers data by key for BatchTupleTSets without grouping data by keys
KeyedGatherUngroupedTLink<K,V>	keyedGatherUngrouped(PartitionFunc<K> partitionFn, java.util.Comparator<K> comparator) Gathers data by key for BatchTupleTSets without grouping data by keys
KeyedPartitionTLink<K,V>	keyedPartition(PartitionFunc<K> partitionFn) Partitions data using a PartitionFunc based on keys
KeyedPipeTLink<K,V>	keyedPipe() Pipe implementation
KeyedReduceTLink<K,V>	keyedReduce(ReduceFunc<V> reduceFn) Reduces data by key for BatchTupleTSets

Methods inherited from class edu.iu.dsc.tws.tset.sets.batch.BatchTupleTSetImpl

addInput, cache, getOutputSchema, getTSetEnv, lazyCache, lazyPersist, persist, setName, withSchema

Methods inherited from class edu.iu.dsc.tws.tset.sets.BaseTSetWithSchema

setOutputSchema

Methods inherited from class edu.iu.dsc.tws.tset.sets.BaseTSet

addChildToGraph, addChildToGraph, equals, getId, getInputs, getName, getParallelism, getStateType, hashCode, isMutable, rename, setMutable, setStateType, setTSetEnv, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.tset.sets.StorableTBase

getDataObject

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName, setName

Methods inherited from interface edu.iu.dsc.tws.tset.sets.BuildableTSet

build

Methods inherited from interface edu.iu.dsc.tws.tset.Buildable

generateID, getTBaseGraph

Field Detail

storedSource

protected KeyedSourceTSet<K,V> storedSource

Method Detail

keyedDirect

public KeyedDirectTLink<K,V> keyedDirect()

Description copied from interface: BatchTupleTSet

Direct/pipe communication

Specified by:

keyedDirect in interface BatchTupleTSet<K,V>

Specified by:

keyedDirect in interface TupleTSet<K,V>

Overrides:

keyedDirect in class BatchTupleTSetImpl<K,V>

Returns:

Keyed Direct TLink

keyedPartition

public KeyedPartitionTLink<K,V> keyedPartition(PartitionFunc<K> partitionFn)

Description copied from interface: BatchTupleTSet

Partitions data using a PartitionFunc based on keys

Specified by:

keyedPartition in interface BatchTupleTSet<K,V>

Specified by:

keyedPartition in interface TupleTSet<K,V>

Overrides:

keyedPartition in class BatchTupleTSetImpl<K,V>

Parameters:

partitionFn - partition function

Returns:

Keyed Partition TLink

keyedPipe

public KeyedPipeTLink<K,V> keyedPipe()

Description copied from interface: BatchTupleTSet

Pipe implementation

Specified by:

keyedPipe in interface BatchTupleTSet<K,V>

Overrides:

keyedPipe in class BatchTupleTSetImpl<K,V>

Returns:

Keyed Pipe TLink

keyedReduce

public KeyedReduceTLink<K,V> keyedReduce(ReduceFunc<V> reduceFn)

Description copied from interface: BatchTupleTSet

Reduces data by key for BatchTupleTSets

Specified by:

keyedReduce in interface BatchTupleTSet<K,V>

Overrides:

keyedReduce in class BatchTupleTSetImpl<K,V>

Parameters:

reduceFn - the reduce function

Returns:

Keyed Reduce TLink

keyedGather

public KeyedGatherTLink<K,V> keyedGather()

Description copied from interface: BatchTupleTSet

Gathers data by key for BatchTupleTSets

Specified by:

keyedGather in interface BatchTupleTSet<K,V>

Overrides:

keyedGather in class BatchTupleTSetImpl<K,V>

Returns:

Keyed Gather TLink

keyedGather

public KeyedGatherTLink<K,V> keyedGather(PartitionFunc<K> partitionFn)

Description copied from interface: BatchTupleTSet

Gathers data by key for BatchTupleTSets

Specified by:

keyedGather in interface BatchTupleTSet<K,V>

Overrides:

keyedGather in class BatchTupleTSetImpl<K,V>

Parameters:

partitionFn - partition function to partition data based on key

Returns:

Keyed Gather TLink

keyedGather

public KeyedGatherTLink<K,V> keyedGather(PartitionFunc<K> partitionFn,
                                         java.util.Comparator<K> comparator)

Description copied from interface: BatchTupleTSet

Gathers data by key for BatchTupleTSets

Specified by:

keyedGather in interface BatchTupleTSet<K,V>

Overrides:

keyedGather in class BatchTupleTSetImpl<K,V>

Parameters:

partitionFn - partition function to partition data based on key

comparator - custom key comparator

Returns:

Keyed Gather TLink

keyedGatherUngrouped

public KeyedGatherUngroupedTLink<K,V> keyedGatherUngrouped()

Description copied from interface: BatchTupleTSet

Gathers data by key for BatchTupleTSets without grouping data by keys

Specified by:

keyedGatherUngrouped in interface BatchTupleTSet<K,V>

Overrides:

keyedGatherUngrouped in class BatchTupleTSetImpl<K,V>

Returns:

this TSet

keyedGatherUngrouped

public KeyedGatherUngroupedTLink<K,V> keyedGatherUngrouped(PartitionFunc<K> partitionFn)

Description copied from interface: BatchTupleTSet

Gathers data by key for BatchTupleTSets without grouping data by keys

Specified by:

keyedGatherUngrouped in interface BatchTupleTSet<K,V>

Overrides:

keyedGatherUngrouped in class BatchTupleTSetImpl<K,V>

Parameters:

partitionFn - partition function to partition data based on key

Returns:

Keyed Gather TLink

keyedGatherUngrouped

public KeyedGatherUngroupedTLink<K,V> keyedGatherUngrouped(PartitionFunc<K> partitionFn,
                                                           java.util.Comparator<K> comparator)

Description copied from interface: BatchTupleTSet

Gathers data by key for BatchTupleTSets without grouping data by keys

Specified by:

keyedGatherUngrouped in interface BatchTupleTSet<K,V>

Overrides:

keyedGatherUngrouped in class BatchTupleTSetImpl<K,V>

Parameters:

partitionFn - partition function to partition data based on key

comparator - custom key comparator

Returns:

Keyed Gather TLink

join

public <VR> JoinTLink<K,V,VR> join(BatchTupleTSet<K,VR> rightTSet,
                                   CommunicationContext.JoinType type,
                                   java.util.Comparator<K> keyComparator,
                                   TaskPartitioner<K> partitioner)

Description copied from interface: BatchTupleTSet

Joins with another BatchTupleTSet. Note that this TSet will be considered the left TSet

Specified by:

join in interface BatchTupleTSet<K,V>

Overrides:

join in class BatchTupleTSetImpl<K,V>

Type Parameters:

VR - value type of the right tset

Parameters:

rightTSet - right tset

type - CommunicationContext.JoinType

keyComparator - key comparator

partitioner - partitioner for keys

Returns:

Joined TLink

join

public <VR> JoinTLink<K,V,VR> join(BatchTupleTSet<K,VR> rightTSet,
                                   CommunicationContext.JoinType type,
                                   java.util.Comparator<K> keyComparator)

Description copied from interface: BatchTupleTSet

Joins with another BatchTupleTSet. Note that this TSet will be considered the left TSet

Specified by:

join in interface BatchTupleTSet<K,V>

Overrides:

join in class BatchTupleTSetImpl<K,V>

Type Parameters:

VR - value type of the right tset

Parameters:

rightTSet - right tset

type - CommunicationContext.JoinType

keyComparator - key comparator

Returns:

Joined TLink

getStoredSourceTSet

public KeyedSourceTSet<K,V> getStoredSourceTSet()

Description copied from interface: StorableTBase

retrieve data saved in the TSet

Specified by:

getStoredSourceTSet in interface StorableTBase<Tuple<K,V>>

Returns:

dataObject

getData

public java.util.List<Tuple<K,V>> getData()

Description copied from interface: StorableTBase

retrieve data saved in the TSet as a List

NOTE: use this method only when you need to pull the data from the tset. Otherwise this would unnecessarily loads data to the memory.

Specified by:

getData in interface StorableTBase<Tuple<K,V>>

Returns:

dataObject

getINode

public INode getINode()

Specified by:

getINode in interface BuildableTSet

getInputSchema

public KeyedSchema getInputSchema()

Overrides:

getInputSchema in class BaseTSetWithSchema<V>