edu.iu.dsc.tws.comms.shuffle

Class FSMerger

java.lang.Object

edu.iu.dsc.tws.comms.shuffle.FSMerger

All Implemented Interfaces:

Shuffle

public class FSMerger
extends java.lang.Object
implements Shuffle

Save the records to file system and retrieve them, this is just values, so no sorting as in the keyed case

Constructor Summary

Constructors

Constructor and Description
FSMerger(long maxBytesInMemory, long maxRecsInMemory, java.lang.String dir, java.lang.String opName, MessageType vType)

Method Summary

Modifier and Type	Method and Description
void	add(byte[] data, int length) Add the data to the file
void	clean() Cleanup the directories
ResettableIterator<java.lang.Object>	readIterator() This method gives the values
void	run() This method saves the data to file system
void	switchToReading() Switch to reading, we cannot add after this

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.comms.shuffle.Shuffle

add, add

Constructor Detail

FSMerger

public FSMerger(long maxBytesInMemory,
                long maxRecsInMemory,
                java.lang.String dir,
                java.lang.String opName,
                MessageType vType)

Method Detail

add

public void add(byte[] data,
                int length)

Add the data to the file

Specified by:

add in interface Shuffle

Parameters:

data - the data as bytes

length - the length of data

switchToReading

public void switchToReading()

Description copied from interface: Shuffle

Switch to reading, we cannot add after this

Specified by:

switchToReading in interface Shuffle

run

public void run()

This method saves the data to file system

Specified by:

run in interface Shuffle

readIterator

public ResettableIterator<java.lang.Object> readIterator()

This method gives the values

Specified by:

readIterator in interface Shuffle

Returns:

an iterator

clean

public void clean()

Cleanup the directories

Specified by:

clean in interface Shuffle