edu.iu.dsc.tws.tset.links.batch

Class BatchTLinkImpl<T1,T0>

java.lang.Object

edu.iu.dsc.tws.tset.links.BaseTLink<T1,T0>

edu.iu.dsc.tws.tset.links.BaseTLinkWithSchema<T1,T0>

edu.iu.dsc.tws.tset.links.batch.BatchTLinkImpl<T1,T0>

All Implemented Interfaces:

BatchTLink<T1,T0>, TLink<T1,T0>, StoringData<T0>, TBase, Buildable, BuildableTLink, java.io.Serializable

Direct Known Subclasses:

BatchGatherLink, BatchIteratorLink, BatchSingleLink, PipeTLink

public abstract class BatchTLinkImpl<T1,T0>
extends BaseTLinkWithSchema<T1,T0>
implements BatchTLink<T1,T0>

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BatchTLinkImpl()

Method Summary

Modifier and Type	Method and Description
StorableTBase<T0>	cache() Runs this TSet and caches the data to an in-memory DataPartition and exposes the data as another TSet.
<P> ComputeTSet<P,T1>	compute(ComputeCollectorFunc<P,T1> computeFunction) Creates a Compute TSet based on the ComputeCollectorFunc provided.
<P> ComputeTSet<P,T1>	compute(ComputeFunc<P,T1> computeFunction) Creates a Compute TSet based on the ComputeFunc provided.
<P> ComputeTSet<P,T1>	compute(java.lang.String n, ComputeCollectorFunc<P,T1> computeFunction)
<P> ComputeTSet<P,T1>	compute(java.lang.String n, ComputeFunc<P,T1> computeFunction)
BatchEnvironment	getTSetEnv() tset env
StorableTBase<T0>	persist() Similar to cache, but the data is stored in a disk based DataPartition.
SinkTSet<T1>	sink(SinkFunc<T1> sinkFunction) Creates a Sink TSet based on the SinkFunc.

Methods inherited from class edu.iu.dsc.tws.tset.links.BaseTLinkWithSchema

getSchema

Methods inherited from class edu.iu.dsc.tws.tset.links.BaseTLink

addChildToGraph, equals, getId, getName, getSourceParallelism, getTargetParallelism, hashCode, rename, setTSetEnv, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.tset.link.batch.BatchTLink

flatmap, forEach, lazyForEach, map, mapToTuple, setName

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName

Methods inherited from interface edu.iu.dsc.tws.api.tset.StoringData

lazyCache, lazyPersist

Methods inherited from interface edu.iu.dsc.tws.tset.links.BuildableTLink

build, getEdge

Methods inherited from interface edu.iu.dsc.tws.tset.Buildable

generateID, getTBaseGraph

Constructor Detail

BatchTLinkImpl

protected BatchTLinkImpl()

Method Detail

getTSetEnv

public BatchEnvironment getTSetEnv()

Description copied from interface: Buildable

tset env

Specified by:

getTSetEnv in interface Buildable

Overrides:

getTSetEnv in class BaseTLink<T1,T0>

Returns:

tset env

compute

public <P> ComputeTSet<P,T1> compute(java.lang.String n,
                                     ComputeFunc<P,T1> computeFunction)

compute

public <P> ComputeTSet<P,T1> compute(java.lang.String n,
                                     ComputeCollectorFunc<P,T1> computeFunction)

compute

public <P> ComputeTSet<P,T1> compute(ComputeFunc<P,T1> computeFunction)

Description copied from interface: TLink

Creates a Compute TSet based on the ComputeFunc provided.

Specified by:

compute in interface BatchTLink<T1,T0>

Specified by:

compute in interface TLink<T1,T0>

Type Parameters:

P - output tset base type

Parameters:

computeFunction - comp function. Takes in T0 type object and map to the output type O

Returns:

Compute TSet

compute

public <P> ComputeTSet<P,T1> compute(ComputeCollectorFunc<P,T1> computeFunction)

Description copied from interface: TLink

Creates a Compute TSet based on the ComputeCollectorFunc provided.

Specified by:

compute in interface BatchTLink<T1,T0>

Specified by:

compute in interface TLink<T1,T0>

Type Parameters:

P - output type (for the RecordCollector)

Parameters:

computeFunction - compute function with collector

Returns:

Compute TSet

sink

public SinkTSet<T1> sink(SinkFunc<T1> sinkFunction)

Description copied from interface: TLink

Creates a Sink TSet based on the SinkFunc.

Specified by:

sink in interface BatchTLink<T1,T0>

Specified by:

sink in interface TLink<T1,T0>

Parameters:

sinkFunction - sink function which takes in <T1>. Similar to a compute, but would not return any TSet

Returns:

Sink tset. This would would be a terminal TSet with no transformation capabilities.

cache

public StorableTBase<T0> cache()

Description copied from interface: StoringData

Runs this TSet and caches the data to an in-memory DataPartition and exposes the data as another TSet.

Specified by:

cache in interface StoringData<T0>

Returns:

Cached TSet

persist

public StorableTBase<T0> persist()

Description copied from interface: StoringData

Similar to cache, but the data is stored in a disk based DataPartition. This method would also expose the checkpointing ability to TSets.

Specified by:

persist in interface StoringData<T0>

Returns:

Persisted / Checkpointed TSets