edu.iu.dsc.tws.examples.batch.terasort

Class TaskPartitionerForRandom

java.lang.Object

edu.iu.dsc.tws.examples.batch.terasort.TaskPartitionerForRandom

All Implemented Interfaces:

TaskPartitioner<byte[]>, java.io.Serializable

Direct Known Subclasses:

TaskPartitionerForSampledData

public class TaskPartitionerForRandom
extends java.lang.Object
implements TaskPartitioner<byte[]>

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected int[]	destinationsList
protected int	keysToOneTask

Constructor Summary

Constructors

Constructor and Description
TaskPartitionerForRandom()

Method Summary

Modifier and Type	Method and Description
void	commit(int source, int partition) Indicate that we are using this partition
protected int	getIndex(byte[] array)
int	partition(int source, byte[] data) Get a partition id, it should be from the set of destinations
void	prepare(java.util.Set<java.lang.Integer> sources, java.util.Set<java.lang.Integer> destinations) Prepare the partition with sources and destinations.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keysToOneTask

protected int keysToOneTask

destinationsList

protected int[] destinationsList

Constructor Detail

TaskPartitionerForRandom

public TaskPartitionerForRandom()

Method Detail

prepare

public void prepare(java.util.Set<java.lang.Integer> sources,
                    java.util.Set<java.lang.Integer> destinations)

Description copied from interface: TaskPartitioner

Prepare the partition with sources and destinations. The sources have values to partition and the partitions should be among destinations

Specified by:

prepare in interface TaskPartitioner<byte[]>

getIndex

protected int getIndex(byte[] array)

partition

public int partition(int source,
                     byte[] data)

Description copied from interface: TaskPartitioner

Get a partition id, it should be from the set of destinations

Specified by:

partition in interface TaskPartitioner<byte[]>

Parameters:

source - source of the data

data - data to be partitioned

Returns:

partition id

commit

public void commit(int source,
                   int partition)

Description copied from interface: TaskPartitioner

Indicate that we are using this partition

Specified by:

commit in interface TaskPartitioner<byte[]>

Parameters:

source - source of the partition

partition - partition to commit