edu.iu.dsc.tws.api.compute.modifiers

Interface Collector

All Known Subinterfaces:

ICollector<T>

All Known Implementing Classes:

CheckpointedSourceOp, ConnectedSink, CSVInputFormatTest.TestSink, DataFileSink, DataInitializationSinkTask, DataObjectSink, DataObjectSink, DataSinkTask, GraphDataSink, IterativeStreamingCompute, IterativeSVMAccuracyReduce, IterativeSVMDataObjectDirectSink, IterativeSVMDataPointReduce, IterativeSVMPrimaryDataObjectDirectSink, IterativeSVMReduce, IterativeSVMWeightVectorObjectDirectSink, IterativeSVMWeightVectorReduce, KMeansComputeJob.KMeansAllReduceTask, KMeansConnectedDataflowExample.KMeansAllReduceTask, KMeansDataObjectDirectSink, PageRankValueHolderSink, PointDataSource, PredictionReduceTask, RowSinkOp, SinkOp, SsspInitialSink, SVMReduce, TaskSchedulerClassTest.TestSink, TeraSort.SamplerReduce

public interface Collector

Special task for collecting the output from tasks

Method Summary

Modifier and Type	Method and Description
default DataPartition<?>	get() Deprecated. use the data partition
default DataPartition<?>	get(java.lang.String name) get the collected valued
default IONames	getCollectibleNames() This method should return a set of collectible names, that can be collected from this collector.

Method Detail

get

@Deprecated
default DataPartition<?> get()

Deprecated. use the data partition

get the collected valued

Returns:

get the default output

get

default DataPartition<?> get(java.lang.String name)

get the collected valued

Parameters:

name - name of the value to collect

Returns:

the partition of the data

getCollectibleNames

default IONames getCollectibleNames()

This method should return a set of collectible names, that can be collected from this collector. These names will be used by task plan builder to cross validate parallelism between two task graphs.

If Collector C of task graph TG1 collects variable "var1" and C's parallelism is n, If Receptor R of task graph TG2 is interested in receiving "var1", R's parallelism should be equal to n and, R should have the same distribution as C among the workers.

Returns:

set of names of collectbles