edu.iu.dsc.tws.examples.utils

Class DataGenerator

java.lang.Object

edu.iu.dsc.tws.examples.utils.DataGenerator

public final class DataGenerator
extends java.lang.Object

Generate a data set

1. We can generate in each worker 2. We can generate in a common location shared by workers, such as HDFS or NFS

Method Summary

Modifier and Type	Method and Description
static void	generateData(java.lang.String type, Path directory, int numOfFiles, int sizeOfFile, int sizeMargin) Generate a data set

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

generateData

public static void generateData(java.lang.String type,
                                Path directory,
                                int numOfFiles,
                                int sizeOfFile,
                                int sizeMargin)
                         throws java.io.IOException

Generate a data set

Parameters:

type - type of file, i.e. csv, text, binary

directory - the directory to generate

numOfFiles - number of files to create

sizeOfFile - size of each file, different types have a different meaning

sizeMargin - size will be varied about this much

Throws:

java.io.IOException