edu.iu.dsc.tws.data.api.splits

Class BinaryInputSplit

java.lang.Object

edu.iu.dsc.tws.data.api.splits.LocatableInputSplit<OT>

edu.iu.dsc.tws.data.api.splits.FileInputSplit<byte[]>

edu.iu.dsc.tws.data.api.splits.BinaryInputSplit

All Implemented Interfaces:

InputSplit<byte[]>, java.io.Serializable

public class BinaryInputSplit
extends FileInputSplit<byte[]>

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class edu.iu.dsc.tws.data.api.splits.FileInputSplit

FileInputSplit.InputSplitOpenThread

Field Summary

Fields

Modifier and Type	Field and Description
protected static int	DEFAULT_RECORD_LENGTH The default value of the record length
protected static java.lang.String	RECORD_LENGTH The configuration key to set the binary record length.
protected int	recordLength The length of a single record in the given binary file.

Fields inherited from class edu.iu.dsc.tws.data.api.splits.FileInputSplit

enumerateNestedFiles, MAX_SPLIT_SIZE_DISCREPANCY, minSplitSize, numSplits, openTimeout, splitLength, splitStart, stream

Constructor Summary

Constructors

Constructor and Description
BinaryInputSplit(int num, Path file, int recordLen, java.lang.String[] hosts)
BinaryInputSplit(int num, Path file, long start, long length, java.lang.String[] hosts) Constructs a split with host information.

Method Summary

Modifier and Type	Method and Description
void	configure(Config parameters) Configures this input format by reading the path to the file from the configuration and setting the record length
int	getBufferSize()
java.nio.ByteOrder	getEndianess()
int	getRecordLength()
byte[]	nextRecord(byte[] record) Reads the next record from the input.
void	open() Opens the given input split.
void	open(Config cfg)
boolean	reachedEnd() Method used to check if the end of the input is reached.
byte[]	readRecord(byte[] reusable, byte[] bytes, int readOffset, int numBytes) Reads a single record from the binary file
void	setBufferSize(int buffSize)
void	setEndianess(java.nio.ByteOrder order)
void	setRecordLength(int recordLen)

Methods inherited from class edu.iu.dsc.tws.data.api.splits.FileInputSplit

close, equals, getLength, getMinSplitSize, getNumSplits, getPath, getStart, hashCode, isEnumerateNestedFiles, setEnumerateNestedFiles, setMinSplitSize, setNumSplits, toString

Methods inherited from class edu.iu.dsc.tws.data.api.splits.LocatableInputSplit

getHostnames, getSplitNumber

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

RECORD_LENGTH

protected static final java.lang.String RECORD_LENGTH

The configuration key to set the binary record length.

See Also:

Constant Field Values

DEFAULT_RECORD_LENGTH

protected static final int DEFAULT_RECORD_LENGTH

The default value of the record length

See Also:

Constant Field Values

recordLength

protected transient int recordLength

The length of a single record in the given binary file.

Constructor Detail

BinaryInputSplit

public BinaryInputSplit(int num,
                        Path file,
                        long start,
                        long length,
                        java.lang.String[] hosts)

Constructs a split with host information.

Parameters:

num - the number of this input split

file - the file name

start - the position of the first byte in the file to process

length - the number of bytes in the file to process (-1 is flag for "read whole file")

hosts - the list of hosts containing the block, possibly null

BinaryInputSplit

public BinaryInputSplit(int num,
                        Path file,
                        int recordLen,
                        java.lang.String[] hosts)

Method Detail

getEndianess

public java.nio.ByteOrder getEndianess()

setEndianess

public void setEndianess(java.nio.ByteOrder order)

getBufferSize

public int getBufferSize()

setBufferSize

public void setBufferSize(int buffSize)

getRecordLength

public int getRecordLength()

setRecordLength

public void setRecordLength(int recordLen)

configure

public void configure(Config parameters)

Configures this input format by reading the path to the file from the configuration and setting the record length

Specified by:

configure in interface InputSplit<byte[]>

Overrides:

configure in class FileInputSplit<byte[]>

Parameters:

parameters - The configuration object to read the parameters from.

open

public void open()
          throws java.io.IOException

Opens the given input split. This method opens the input stream to the specified file, allocates read buffers and positions the stream at the correct position, making sure that any partial record at the beginning is skipped.

Specified by:

open in interface InputSplit<byte[]>

Overrides:

open in class FileInputSplit<byte[]>

Throws:

java.io.IOException - Thrown, if the spit could not be opened due to an I/O problem.

open

public void open(Config cfg)
          throws java.io.IOException

Specified by:

open in interface InputSplit<byte[]>

Overrides:

open in class FileInputSplit<byte[]>

Throws:

java.io.IOException

reachedEnd

public boolean reachedEnd()

Description copied from interface: InputSplit

Method used to check if the end of the input is reached.

When this method is called, the input format it guaranteed to be opened.

Returns:

True if the end is reached, otherwise false.

nextRecord

public byte[] nextRecord(byte[] record)
                  throws java.io.IOException

Description copied from interface: InputSplit

Reads the next record from the input.

When this method is called, the input format it guaranteed to be opened.

Parameters:

record - Object that may be reused.

Returns:

Read record.

Throws:

java.io.IOException - Thrown, if an I/O error occurred.

readRecord

public byte[] readRecord(byte[] reusable,
                         byte[] bytes,
                         int readOffset,
                         int numBytes)
                  throws java.io.IOException

Reads a single record from the binary file

Throws:

java.io.IOException