public final class DataGenerator
extends java.lang.Object
1. We can generate in each worker 2. We can generate in a common location shared by workers, such as HDFS or NFS
Modifier and Type | Method and Description |
---|---|
static void |
generateData(java.lang.String type,
Path directory,
int numOfFiles,
int sizeOfFile,
int sizeMargin)
Generate a data set
|
public static void generateData(java.lang.String type, Path directory, int numOfFiles, int sizeOfFile, int sizeMargin) throws java.io.IOException
type
- type of file, i.e. csv, text, binarydirectory
- the directory to generatenumOfFiles
- number of files to createsizeOfFile
- size of each file, different types have a different meaningsizeMargin
- size will be varied about this muchjava.io.IOException