edu.iu.dsc.tws.examples.ml.svm.job

Class SvmSgdAdvancedRunner

java.lang.Object

edu.iu.dsc.tws.task.impl.TaskWorker

edu.iu.dsc.tws.examples.ml.svm.job.SvmSgdAdvancedRunner

All Implemented Interfaces:

IWorker

public class SvmSgdAdvancedRunner
extends TaskWorker

Field Summary

Fields inherited from class edu.iu.dsc.tws.task.impl.TaskWorker

computeEnvironment, config, persistentVolume, taskExecutor, volatileVolume, workerController, workerEnvironment, workerId

Constructor Summary

Constructors

Constructor and Description
SvmSgdAdvancedRunner()

Method Summary

Modifier and Type	Method and Description
void	execute() A user needs to implement this method to create the task graph and execute it
DataObject<double[]>	executeIterativeTrainingGraph() This method executes the iterative training graph Training is done in parallel depending on the parallelism factor given In this implementation the data loading parallelism and data computing or training parallelism is same.
DataObject<java.lang.Object>	executeTestingDataLoadingTaskGraph() This method loads the testing data The loaded test data is used to evaluate the trained data Testing data is loaded in parallel depending on the parallelism parameter given There are partitions created equal to the parallelism Later this will be used to do the testing in parallel in the testing task graph
DataObject<java.lang.Object>	executeTestingTaskGraph() This method executes the testing taskgraph with testing data loaded from testing taskgraph and uses the final weight vector obtained from the training task graph Testing is also done in a parallel way.
DataObject<java.lang.Object>	executeTrainingDataLoadingTaskGraph() This method loads the training data in a distributed mode dataStreamerParallelism is the amount of parallelism used in loaded the data in parallel.
DataObject<double[]>	executeTrainingGraph() This method executes the training graph Training is done in parallel depending on the parallelism factor given In this implementation the data loading parallelism and data computing or training parallelism is same.
DataObject<java.lang.Object>	executeWeightVectorLoadingTaskGraph() This method loads the training data in a distributed mode dataStreamerParallelism is the amount of parallelism used in loaded the data in parallel.
void	initializeExecute() Initializing the execute method
void	initializeParameters() This method initializes the parameters in running SVM
void	printTaskSummary()
DataObject<java.lang.Object>	retrieveTestingAccuracyObject(ComputeGraph predictionGraph, ExecutionPlan predictionPlan) This method retrieves the accuracy data object from the prediction task graph
DataObject<double[]>	retrieveWeightVectorFromTaskGraph(ComputeGraph graph1, ExecutionPlan plan1) This method returns the final weight vector from the trained model
double	retriveFinalTestingAccuracy(DataObject<java.lang.Object> finalRes) Calculates the final accuracy by taking the dataParallelism in to consideration Here the parallelism is vital as we need to know the average accuracy produced by each testing data set.
void	saveResults()

Methods inherited from class edu.iu.dsc.tws.task.impl.TaskWorker

execute

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SvmSgdAdvancedRunner

public SvmSgdAdvancedRunner()

Method Detail

execute

public void execute()

Description copied from class: TaskWorker

A user needs to implement this method to create the task graph and execute it

Specified by:

execute in class TaskWorker

initializeParameters

public void initializeParameters()

This method initializes the parameters in running SVM

initializeExecute

public void initializeExecute()

Initializing the execute method

executeTrainingDataLoadingTaskGraph

public DataObject<java.lang.Object> executeTrainingDataLoadingTaskGraph()

This method loads the training data in a distributed mode dataStreamerParallelism is the amount of parallelism used in loaded the data in parallel.

Returns:

twister2 DataObject containing the training data

executeWeightVectorLoadingTaskGraph

public DataObject<java.lang.Object> executeWeightVectorLoadingTaskGraph()

This method loads the training data in a distributed mode dataStreamerParallelism is the amount of parallelism used in loaded the data in parallel.

Returns:

twister2 DataObject containing the training data

executeTestingDataLoadingTaskGraph

public DataObject<java.lang.Object> executeTestingDataLoadingTaskGraph()

This method loads the testing data The loaded test data is used to evaluate the trained data Testing data is loaded in parallel depending on the parallelism parameter given There are partitions created equal to the parallelism Later this will be used to do the testing in parallel in the testing task graph

Returns:

twister2 DataObject containing the testing data

executeTrainingGraph

public DataObject<double[]> executeTrainingGraph()

This method executes the training graph Training is done in parallel depending on the parallelism factor given In this implementation the data loading parallelism and data computing or training parallelism is same. It is the general model to keep them equal. But you can increase the parallelism the way you want. But it is adviced to keep these values equal. Dynamic parallelism in training is not yet tested fully in Twister2 Framework.

Returns:

Twister2 DataObject<double[]> containing the reduced weight vector

executeIterativeTrainingGraph

public DataObject<double[]> executeIterativeTrainingGraph()

This method executes the iterative training graph Training is done in parallel depending on the parallelism factor given In this implementation the data loading parallelism and data computing or training parallelism is same. It is the general model to keep them equal. But you can increase the parallelism the way you want. But it is adviced to keep these values equal. Dynamic parallelism in training is not yet tested fully in Twister2 Framework.

Returns:

Twister2 DataObject<double[]> containing the reduced weight vector

retrieveWeightVectorFromTaskGraph

public DataObject<double[]> retrieveWeightVectorFromTaskGraph(ComputeGraph graph1,
                                                              ExecutionPlan plan1)

This method returns the final weight vector from the trained model

Parameters:

graph1 - DataflowTaskGraph from which we retrieve the final weight vector

plan1 - ExecutionPlan from which we retrive the final weight vector

Returns:

Twister2 DataObject containing the final reduced weight vector is retrieved

executeTestingTaskGraph

public DataObject<java.lang.Object> executeTestingTaskGraph()

This method executes the testing taskgraph with testing data loaded from testing taskgraph and uses the final weight vector obtained from the training task graph Testing is also done in a parallel way. At the testing data loading stage we load the data in parallel with reference to the given parallelism and testing is also in in parallel Then we get test results for all these testing data partitions

Returns:

Returns the Accuracy value obtained

retrieveTestingAccuracyObject

public DataObject<java.lang.Object> retrieveTestingAccuracyObject(ComputeGraph predictionGraph,
                                                                  ExecutionPlan predictionPlan)

This method retrieves the accuracy data object from the prediction task graph

Parameters:

predictionGraph - ComputeGraph from which the final accuracy is retrieved

predictionPlan - PredictionTaskGraph from which the final accuracy is retrieved

Returns:

returns the Twister2 DataObject containing the accuracy object

retriveFinalTestingAccuracy

public double retriveFinalTestingAccuracy(DataObject<java.lang.Object> finalRes)

Calculates the final accuracy by taking the dataParallelism in to consideration Here the parallelism is vital as we need to know the average accuracy produced by each testing data set.

Parameters:

finalRes - DataObject which contains the final accuracy

printTaskSummary

public void printTaskSummary()

saveResults

public void saveResults()
                 throws java.io.IOException

Throws:

java.io.IOException