edu.iu.dsc.tws.tset.env

Class StreamingEnvironment

java.lang.Object

edu.iu.dsc.tws.tset.env.TSetEnvironment

edu.iu.dsc.tws.tset.env.StreamingEnvironment

Direct Known Subclasses:

StreamTSetCDFWEnvironment

public class StreamingEnvironment
extends TSetEnvironment

Implementation of TSetEnvironment for streaming OperationMode.

There is only a single execution mode that would run the entire TSet graph.

Constructor Summary

Constructors

Constructor and Description
StreamingEnvironment()
StreamingEnvironment(WorkerEnvironment wEnv)

Method Summary

Modifier and Type	Method and Description
BaseTSet<java.lang.Integer>	createArrowSource(java.lang.String filePath, int parallelism)
SSourceTSet<java.lang.Object>	createArrowSource(java.lang.String filePath, java.lang.String fileName, int parallelism, java.lang.String schema)
SSourceTSet<java.lang.String[]>	createCSVSource(java.lang.String filePath, int datasize, int parallelism, java.lang.String type)
<K,V> SKeyedSourceTSet<K,V>	createKeyedSource(SourceFunc<Tuple<K,V>> source, int parallelism) Creates a Keyed Source TSet based on the SourceFunc that produces a Tuple
<K,V> SKeyedSourceTSet<K,V>	createKeyedSource(java.lang.String name, SourceFunc<Tuple<K,V>> source, int parallelism) Same as above, but a source tset name can be provided at the instantiation
<T> SSourceTSet<T>	createSource(SourceFunc<T> source, int parallelism) Creates a source TSet based on the SourceFunc
<T> SSourceTSet<T>	createSource(java.lang.String name, SourceFunc<T> source, int parallelism) Same as above, but a source tset name can be provided at the instantiation
SSourceTSet<java.lang.String>	createTextSource(java.lang.String filePath, int dataSize, int parallelism, java.lang.String type)
OperationMode	getOperationMode() Returns the OperationMode
void	run() Runs the entire TSet graph

Methods inherited from class edu.iu.dsc.tws.tset.env.TSetEnvironment

addInput, close, executeBuildContext, getConfig, getDefaultParallelism, getGraph, getInputs, getNoOfWorkers, getTaskExecutor, getTSetGraph, getWorkerEnv, getWorkerID, initBatch, initCheckpointing, initStreaming, isCheckpointingEnabled, parallelize, parallelize, parallelize, parallelize, setDefaultParallelism, settBaseGraph

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamingEnvironment

public StreamingEnvironment(WorkerEnvironment wEnv)

StreamingEnvironment

public StreamingEnvironment()

Method Detail

getOperationMode

public OperationMode getOperationMode()

Description copied from class: TSetEnvironment

Returns the OperationMode

Specified by:

getOperationMode in class TSetEnvironment

Returns:

operation mode

createSource

public <T> SSourceTSet<T> createSource(SourceFunc<T> source,
                                       int parallelism)

Description copied from class: TSetEnvironment

Creates a source TSet based on the SourceFunc

Specified by:

createSource in class TSetEnvironment

Type Parameters:

T - data type

Parameters:

source - source function

parallelism - parallelism

Returns:

Source TSet

createSource

public <T> SSourceTSet<T> createSource(java.lang.String name,
                                       SourceFunc<T> source,
                                       int parallelism)

Description copied from class: TSetEnvironment

Same as above, but a source tset name can be provided at the instantiation

Specified by:

createSource in class TSetEnvironment

Type Parameters:

T - data type

Parameters:

name - name for the tset

source - source function

parallelism - parallelism

Returns:

Source TSet

createCSVSource

public SSourceTSet<java.lang.String[]> createCSVSource(java.lang.String filePath,
                                                       int datasize,
                                                       int parallelism,
                                                       java.lang.String type)

Specified by:

createCSVSource in class TSetEnvironment

createTextSource

public SSourceTSet<java.lang.String> createTextSource(java.lang.String filePath,
                                                      int dataSize,
                                                      int parallelism,
                                                      java.lang.String type)

Specified by:

createTextSource in class TSetEnvironment

createArrowSource

public BaseTSet<java.lang.Integer> createArrowSource(java.lang.String filePath,
                                                     int parallelism)

Specified by:

createArrowSource in class TSetEnvironment

createArrowSource

public SSourceTSet<java.lang.Object> createArrowSource(java.lang.String filePath,
                                                       java.lang.String fileName,
                                                       int parallelism,
                                                       java.lang.String schema)

createKeyedSource

public <K,V> SKeyedSourceTSet<K,V> createKeyedSource(SourceFunc<Tuple<K,V>> source,
                                                     int parallelism)

Description copied from class: TSetEnvironment

Creates a Keyed Source TSet based on the SourceFunc that produces a Tuple

Specified by:

createKeyedSource in class TSetEnvironment

Type Parameters:

K - key type

V - value type

Parameters:

source - source function

parallelism - parallelism

Returns:

Keyed Source TSet

createKeyedSource

public <K,V> SKeyedSourceTSet<K,V> createKeyedSource(java.lang.String name,
                                                     SourceFunc<Tuple<K,V>> source,
                                                     int parallelism)

Description copied from class: TSetEnvironment

Same as above, but a source tset name can be provided at the instantiation

Specified by:

createKeyedSource in class TSetEnvironment

Type Parameters:

K - key type

V - value type

Parameters:

name - name for the tset

source - source function

parallelism - parallelism

Returns:

Keyed Source TSet

run

public void run()

Runs the entire TSet graph