edu.iu.dsc.tws.data.api.out

Class FileOutputWriter<T>

java.lang.Object

edu.iu.dsc.tws.data.api.out.FileOutputWriter<T>

Type Parameters:

T - the type of data to write

All Implemented Interfaces:

OutputWriter<T>, java.io.Serializable

Direct Known Subclasses:

CSVOutputWriter, TextOutputWriter

public abstract class FileOutputWriter<T>
extends java.lang.Object
implements OutputWriter<T>

Abstract class for writing to file system

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected FileSystem	fs File system object
protected java.util.Map<java.lang.Integer,FSDataOutputStream>	openStreams Opened streams
protected Path	outPath File output path
protected java.io.PrintWriter	pw
protected FileSystem.WriteMode	writeMode Write mode of the files

Constructor Summary

Constructors

Constructor and Description
FileOutputWriter(FileSystem.WriteMode writeMode, Path outPath)
FileOutputWriter(FileSystem.WriteMode writeMode, Path outPath, Config config)

Method Summary

Modifier and Type	Method and Description
void	close() Close the writer, flush everything
void	configure(Config config) Configure the output writer
protected abstract void	createOutput(int partition, FSDataOutputStream out) Create a suitable output
static java.lang.String	generateRandom(int length)
FileSystem.WriteMode	getWriteMode()
void	setWriteMode(FileSystem.WriteMode writeMode)
void	write(int partition, T out) Add value to output
void	write(T out)
protected abstract void	writeRecord(int partition, T data) Write the record to output
protected abstract void	writeRecord(T data)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fs

protected FileSystem fs

File system object

pw

protected java.io.PrintWriter pw

openStreams

protected java.util.Map<java.lang.Integer,FSDataOutputStream> openStreams

Opened streams

writeMode

protected FileSystem.WriteMode writeMode

Write mode of the files

outPath

protected Path outPath

File output path

Constructor Detail

FileOutputWriter

public FileOutputWriter(FileSystem.WriteMode writeMode,
                        Path outPath)

FileOutputWriter

public FileOutputWriter(FileSystem.WriteMode writeMode,
                        Path outPath,
                        Config config)

Method Detail

getWriteMode

public FileSystem.WriteMode getWriteMode()

setWriteMode

public void setWriteMode(FileSystem.WriteMode writeMode)

write

public void write(int partition,
                  T out)

Description copied from interface: OutputWriter

Add value to output

Specified by:

write in interface OutputWriter<T>

write

public void write(T out)

Specified by:

write in interface OutputWriter<T>

generateRandom

public static java.lang.String generateRandom(int length)

createOutput

protected abstract void createOutput(int partition,
                                     FSDataOutputStream out)

Create a suitable output

Parameters:

partition - partition id

out - the out stream

writeRecord

protected abstract void writeRecord(int partition,
                                    T data)

Write the record to output

Parameters:

partition - partition id

data - data

writeRecord

protected abstract void writeRecord(T data)

configure

public void configure(Config config)

Description copied from interface: OutputWriter

Configure the output writer

Specified by:

configure in interface OutputWriter<T>

Parameters:

config - configuration

close

public void close()

Description copied from interface: OutputWriter

Close the writer, flush everything

Specified by:

close in interface OutputWriter<T>