edu.iu.dsc.tws.tset.sets.batch.row

Class BatchRowTSetImpl

java.lang.Object

edu.iu.dsc.tws.tset.sets.BaseTSet<T>

edu.iu.dsc.tws.tset.sets.BaseTSetWithSchema<Row>

edu.iu.dsc.tws.tset.sets.batch.row.BatchRowTSetImpl

All Implemented Interfaces:

AcceptingData<Row>, BatchRowTSet, StoringData<Row>, TBase, Buildable, BuildableTSet, java.io.Serializable

Direct Known Subclasses:

RowComputeTSet, RowSourceTSet, RowStoredTSet

public abstract class BatchRowTSetImpl
extends BaseTSetWithSchema<Row>
implements BatchRowTSet

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class edu.iu.dsc.tws.tset.sets.BaseTSet

BaseTSet.StateType

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BatchRowTSetImpl()
protected	BatchRowTSetImpl(BatchEnvironment tSetEnv, java.lang.String name, int parallelism, Schema inputSchema)

Method Summary

Modifier and Type	Method and Description
TBase	addInput(java.lang.String key, StorableTBase<?> input) Allows users to pass in other TSets as inputs for a TSet
StorableTBase<Row>	cache() Runs this TSet and caches the data to an in-memory DataPartition and exposes the data as another TSet.
BatchRowTLink	direct() Direct/pipe communication
BatchEnvironment	getTSetEnv() tset env
BatchRowTLink	join(BatchRowTSet rightTSet, CommunicationContext.JoinType type, java.util.Comparator<Row> keyComparator) Joins with another BatchTupleTSet.
StorableTBase<Row>	lazyCache() Performs caching lazily.
StorableTBase<Row>	lazyPersist() Performs persisting lazily.
BatchRowTLink	partition(PartitionFunc<Row> partitionFn, int targetParallelism, int column) Returns a Partition TLink that would partition data according based on a function provided.
StorableTBase<Row>	persist() Similar to cache, but the data is stored in a disk based DataPartition.
BatchRowTLink	pipe()
BatchRowTSetImpl	setName(java.lang.String n) Sets the name for the TBase
BatchRowTSet	withSchema(RowSchema schema) Sets the data type of the TSet output.

Methods inherited from class edu.iu.dsc.tws.tset.sets.BaseTSetWithSchema

getInputSchema, getOutputSchema, setOutputSchema

Methods inherited from class edu.iu.dsc.tws.tset.sets.BaseTSet

addChildToGraph, addChildToGraph, equals, getId, getInputs, getName, getParallelism, getStateType, hashCode, isMutable, rename, setMutable, setStateType, setTSetEnv, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.tset.TBase

getId, getName

Methods inherited from interface edu.iu.dsc.tws.tset.sets.BuildableTSet

build, getINode

Methods inherited from interface edu.iu.dsc.tws.tset.Buildable

generateID, getTBaseGraph

Constructor Detail

BatchRowTSetImpl

protected BatchRowTSetImpl(BatchEnvironment tSetEnv,
                           java.lang.String name,
                           int parallelism,
                           Schema inputSchema)

BatchRowTSetImpl

protected BatchRowTSetImpl()

Method Detail

partition

public BatchRowTLink partition(PartitionFunc<Row> partitionFn,
                               int targetParallelism,
                               int column)

Description copied from interface: BatchRowTSet

Returns a Partition TLink that would partition data according based on a function provided. The parallelism of the target TSet can also be specified.

Specified by:

partition in interface BatchRowTSet

Parameters:

partitionFn - Partition function

targetParallelism - column index to use

column - Target parallelism

Returns:

Partition TLink

join

public BatchRowTLink join(BatchRowTSet rightTSet,
                          CommunicationContext.JoinType type,
                          java.util.Comparator<Row> keyComparator)

Description copied from interface: BatchRowTSet

Joins with another BatchTupleTSet. Note that this TSet will be considered the left TSet

Specified by:

join in interface BatchRowTSet

Parameters:

rightTSet - right tset

type - CommunicationContext.JoinType

keyComparator - key comparator

Returns:

Joined TLink

getTSetEnv

public BatchEnvironment getTSetEnv()

Description copied from interface: Buildable

tset env

Specified by:

getTSetEnv in interface Buildable

Overrides:

getTSetEnv in class BaseTSet<Row>

Returns:

tset env

direct

public BatchRowTLink direct()

Description copied from interface: BatchRowTSet

Direct/pipe communication

Specified by:

direct in interface BatchRowTSet

Returns:

Keyed Direct TLink

pipe

public BatchRowTLink pipe()

setName

public BatchRowTSetImpl setName(java.lang.String n)

Description copied from interface: TBase

Sets the name for the TBase

Specified by:

setName in interface TBase

cache

public StorableTBase<Row> cache()

Description copied from interface: StoringData

Runs this TSet and caches the data to an in-memory DataPartition and exposes the data as another TSet.

Specified by:

cache in interface StoringData<Row>

Returns:

Cached TSet

lazyCache

public StorableTBase<Row> lazyCache()

Description copied from interface: StoringData

Performs caching lazily. i.e. cache operation would only be performed when the TSet is evaluated explicitly.

Specified by:

lazyCache in interface StoringData<Row>

Returns:

Cached TSet

persist

public StorableTBase<Row> persist()

Description copied from interface: StoringData

Similar to cache, but the data is stored in a disk based DataPartition. This method would also expose the checkpointing ability to TSets.

Specified by:

persist in interface StoringData<Row>

Returns:

Persisted / Checkpointed TSets

lazyPersist

public StorableTBase<Row> lazyPersist()

Description copied from interface: StoringData

Performs persisting lazily.

Specified by:

lazyPersist in interface StoringData<Row>

Returns:

Persisted / Checkpointed TSets

addInput

public TBase addInput(java.lang.String key,
                      StorableTBase<?> input)

Description copied from interface: AcceptingData

Allows users to pass in other TSets as inputs for a TSet

Specified by:

addInput in interface AcceptingData<Row>

Parameters:

key - the key used to store the given TSet

input - a StorableTBase TSet to be added as an input

Returns:

this TSet

withSchema

public BatchRowTSet withSchema(RowSchema schema)

Description copied from interface: BatchRowTSet

Sets the data type of the TSet output. This will be used in the packers for efficient SER-DE operations in the following TLinks

Specified by:

withSchema in interface BatchRowTSet

Parameters:

schema - data type as a MessageType

Returns:

this TSet