edu.iu.dsc.tws.examples.batch.kmeans

Class KMeansComputeJob

java.lang.Object

edu.iu.dsc.tws.examples.batch.kmeans.KMeansComputeJob

All Implemented Interfaces:

Twister2Worker

public class KMeansComputeJob
extends java.lang.Object
implements Twister2Worker

It is the main class for the K-Means clustering which consists of four main tasks namely generation of datapoints and centroids, partition and read the partitioned data points, read the centroids, and finally perform the distance calculation.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	KMeansComputeJob.CentroidAggregator This class aggregates the cluster centroid values and sum the new centroid values.
static class	KMeansComputeJob.KMeansAllReduceTask
static class	KMeansComputeJob.KMeansSourceTask

Constructor Summary

Constructors

Constructor and Description
KMeansComputeJob()

Method Summary

Modifier and Type	Method and Description
static ComputeGraph	buildCentroidsTG(java.lang.String centroidDirectory, int csize, int parallelismValue, int dimension, Config conf, java.lang.String filetype)
static ComputeGraph	buildDataPointsTG(java.lang.String dataDirectory, int dsize, int parallelismValue, int dimension, Config conf, java.lang.String filetype)
static ComputeGraph	buildKMeansTG(int parallelismValue, Config conf)
void	execute(WorkerEnvironment workerEnv) First, the execute method invokes the generateDataPoints method to generate the datapoints file and centroid file based on the respective filesystem submitted by the user.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KMeansComputeJob

public KMeansComputeJob()

Method Detail

execute

public void execute(WorkerEnvironment workerEnv)

First, the execute method invokes the generateDataPoints method to generate the datapoints file and centroid file based on the respective filesystem submitted by the user. Next, it invoke the DataObjectSource and DataObjectSink to partition and read the partitioned data points respectively through data points task graph. Then, it calls the DataFileReader to read the centroid values from the filesystem through centroid task graph. Next, the datapoints are stored in DataSet \(0th object\) and centroids are stored in DataSet 1st object\). Finally, it constructs the kmeans task graph to perform the clustering process which computes the distance between the centroids and data points.

Specified by:

execute in interface Twister2Worker

buildDataPointsTG

public static ComputeGraph buildDataPointsTG(java.lang.String dataDirectory,
                                             int dsize,
                                             int parallelismValue,
                                             int dimension,
                                             Config conf,
                                             java.lang.String filetype)

buildCentroidsTG

public static ComputeGraph buildCentroidsTG(java.lang.String centroidDirectory,
                                            int csize,
                                            int parallelismValue,
                                            int dimension,
                                            Config conf,
                                            java.lang.String filetype)

buildKMeansTG

public static ComputeGraph buildKMeansTG(int parallelismValue,
                                         Config conf)