edu.iu.dsc.tws.tset.links.batch

Class PipeTLink<T>

java.lang.Object

edu.iu.dsc.tws.tset.links.BaseTLink<T1,T0>

edu.iu.dsc.tws.tset.links.BaseTLinkWithSchema<T1,T0>

edu.iu.dsc.tws.tset.links.batch.BatchTLinkImpl<T,T>

edu.iu.dsc.tws.tset.links.batch.PipeTLink<T>

All Implemented Interfaces:

BatchTLink<T,T>, TLink<T,T>, StoringData<T>, TBase, Buildable, BuildableTLink, java.io.Serializable

public class PipeTLink<T>
extends BatchTLinkImpl<T,T>

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
PipeTLink(BatchEnvironment tSetEnv, int sourceParallelism, Schema schema)
PipeTLink(BatchEnvironment tSetEnv, java.lang.String name, int sourceParallelism, Schema schema)

Method Summary

Modifier and Type	Method and Description
CachedTSet<T>	cache() Runs this TSet and caches the data to an in-memory DataPartition and exposes the data as another TSet.
<O> ComputeTSet<O,T>	flatmap(FlatMapFunc<O,T> mapFn) Performs flat map operation based on the FlatMapFunc provided
void	forEach(ApplyFunc<T> applyFunction) Applies a function elementwise.
Edge	getEdge()
CachedTSet<T>	lazyCache() Performs caching lazily.
ComputeTSet<java.lang.Object,T>	lazyForEach(ApplyFunc<T> applyFunction)
PersistedTSet<T>	lazyPersist() Performs persisting lazily.
<O> ComputeTSet<O,T>	map(MapFunc<O,T> mapFn) Performs elementwise map operation based on the MapFunc provided
<K,V> KeyedTSet<K,V>	mapToTuple(MapFunc<Tuple<K,V>,T> genTupleFn) Maps the data passed through the TLink to Tuple based on a MapFunc and thereby creating a TupleTSet
PersistedTSet<T>	persist() Similar to cache, but the data is stored in a disk based DataPartition.
PipeTLink<T>	setName(java.lang.String name) Name of the TSet and return the same tlink

Methods inherited from class edu.iu.dsc.tws.tset.links.batch.BatchTLinkImpl

compute, compute, compute, compute, getTSetEnv, sink

Methods inherited from class edu.iu.dsc.tws.tset.links.BaseTLinkWithSchema

getSchema

Methods inherited from class edu.iu.dsc.tws.tset.links.BaseTLink

addChildToGraph, equals, getId, getName, getSourceParallelism, getTargetParallelism, hashCode, rename, setTSetEnv, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName

Methods inherited from interface edu.iu.dsc.tws.tset.links.BuildableTLink

build

Methods inherited from interface edu.iu.dsc.tws.tset.Buildable

generateID, getTBaseGraph

Constructor Detail

PipeTLink

public PipeTLink(BatchEnvironment tSetEnv,
                 int sourceParallelism,
                 Schema schema)

PipeTLink

public PipeTLink(BatchEnvironment tSetEnv,
                 java.lang.String name,
                 int sourceParallelism,
                 Schema schema)

Method Detail

setName

public PipeTLink<T> setName(java.lang.String name)

Description copied from interface: TLink

Name of the TSet and return the same tlink

Parameters:

name - name

Returns:

this tlink

forEach

public void forEach(ApplyFunc<T> applyFunction)

Description copied from interface: TLink

Applies a function elementwise. Similar to compute, but the ApplyFunc does not return anything.

Parameters:

applyFunction - apply function

lazyForEach

public ComputeTSet<java.lang.Object,T> lazyForEach(ApplyFunc<T> applyFunction)

mapToTuple

public <K,V> KeyedTSet<K,V> mapToTuple(MapFunc<Tuple<K,V>,T> genTupleFn)

Description copied from interface: TLink

Maps the data passed through the TLink to Tuple based on a MapFunc and thereby creating a TupleTSet

Type Parameters:

K - key type

V - value type

Parameters:

genTupleFn - MapFunc to generate a Tuple

Returns:

Keyed TSet

flatmap

public <O> ComputeTSet<O,T> flatmap(FlatMapFunc<O,T> mapFn)

Description copied from interface: TLink

Performs flat map operation based on the FlatMapFunc provided

Type Parameters:

O - map function to T0 to multiple elements of <O>

Parameters:

mapFn - map function which can produce multiple elements for a single <T0> element

Returns:

Compute TSet

map

public <O> ComputeTSet<O,T> map(MapFunc<O,T> mapFn)

Description copied from interface: TLink

Performs elementwise map operation based on the MapFunc provided

Type Parameters:

O - output type

Parameters:

mapFn - map function T0 to O

Returns:

Compute tset

getEdge

public Edge getEdge()

lazyCache

public CachedTSet<T> lazyCache()

Description copied from interface: StoringData

Performs caching lazily. i.e. cache operation would only be performed when the TSet is evaluated explicitly.

Returns:

Cached TSet

lazyPersist

public PersistedTSet<T> lazyPersist()

Description copied from interface: StoringData

Performs persisting lazily.

Returns:

Persisted / Checkpointed TSets

cache

public CachedTSet<T> cache()

Description copied from interface: StoringData

Runs this TSet and caches the data to an in-memory DataPartition and exposes the data as another TSet.

Specified by:

cache in interface StoringData<T>

Overrides:

cache in class BatchTLinkImpl<T,T>

Returns:

Cached TSet

persist

public PersistedTSet<T> persist()

Description copied from interface: StoringData

Similar to cache, but the data is stored in a disk based DataPartition. This method would also expose the checkpointing ability to TSets.

Specified by:

persist in interface StoringData<T>

Overrides:

persist in class BatchTLinkImpl<T,T>

Returns:

Persisted / Checkpointed TSets