edu.iu.dsc.tws.tset.sources

Class HadoopSource<K,V,F extends <any>>

java.lang.Object

edu.iu.dsc.tws.tset.sources.HadoopSource<K,V,F>

Type Parameters:

K - Key

V - Value

F - InputFormat

All Implemented Interfaces:

SourceFunc<Tuple<K,V>>, TFunction<Tuple<K,V>,Tuple<K,V>>, java.io.Serializable

public class HadoopSource<K,V,F extends <any>>
extends java.lang.Object
implements SourceFunc<Tuple<K,V>>

A Hadoop source for reading the values. If this source is used, we need to make sure the values returned by it can be used throughout the computations. The default Hadoop input formatters clear the values immediately after they are returned.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected <any>	currentReader The current record reader

Constructor Summary

Constructors

Constructor and Description
HadoopSource(Configuration conf, java.lang.Class<F> inputClazz)

Method Summary

Modifier and Type	Method and Description
boolean	hasNext() Weather we have more data
Tuple<K,V>	next() Get the next value
void	prepare(TSetContext ctx) Prepare the function.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.tset.fn.SourceFunc

close

Methods inherited from interface edu.iu.dsc.tws.api.tset.fn.TFunction

end

Field Detail

currentReader

protected <any> currentReader

The current record reader

Constructor Detail

HadoopSource

public HadoopSource(Configuration conf,
                    java.lang.Class<F> inputClazz)

Method Detail

prepare

public void prepare(TSetContext ctx)

Description copied from interface: TFunction

Prepare the function. Use this context in case you need to add/get inputs

Specified by:

prepare in interface TFunction<Tuple<K,V>,Tuple<K,V>>

Parameters:

ctx - CONTEXT

hasNext

public boolean hasNext()

Description copied from interface: SourceFunc

Weather we have more data

Specified by:

hasNext in interface SourceFunc<Tuple<K,V>>

Returns:

true if there is more data to be read

next

public Tuple<K,V> next()

Description copied from interface: SourceFunc

Get the next value

Specified by:

next in interface SourceFunc<Tuple<K,V>>

Returns:

the next value