edu.iu.dsc.tws.comms.dfw.io.reduce.keyed

Class KReduceStreamingReceiver

java.lang.Object

edu.iu.dsc.tws.comms.dfw.io.KeyedReceiver

edu.iu.dsc.tws.comms.dfw.io.reduce.keyed.KReduceStreamingReceiver

All Implemented Interfaces:

MessageReceiver

Direct Known Subclasses:

KReduceStreamingFinalReceiver, KReduceStreamingPartialReceiver

public abstract class KReduceStreamingReceiver
extends KeyedReceiver

Abstract class that is extended by keyed reduce streaming receivers

Field Summary

Fields

Modifier and Type	Field and Description
protected int	localWindowCount variable used to keep track of the current local window count.
protected ReduceFunction	reduceFunction The function that is used for the reduce operation
protected int	windowSize Streaming messages are only kept until the window size is met.

Fields inherited from class edu.iu.dsc.tws.comms.dfw.io.KeyedReceiver

batchDone, dataFlowOperation, destination, executor, finishedSources, isEmptySent, isFinalBatchReceiver, keyLimit, limitPerKey, messages, representSource, representSourceSet, sendQueue, thisSources

Constructor Summary

Constructors

Constructor and Description
KReduceStreamingReceiver()

Method Summary

Modifier and Type	Method and Description
protected boolean	moveMessagesToSendQueue(int target, java.util.Map<java.lang.Object,java.util.Queue<java.lang.Object>> messagesPerTarget) moves all the buffered messages into the sendQueue for the given target, this method assumes that for each target that there is only one object in the queue.
protected boolean	offerMessage(int target, java.lang.Object object) The reduce operation overrides the offer method because the reduce operation does not save all the incoming messages, it rather reduces messages with the same key and saves only the reduced values.

Methods inherited from class edu.iu.dsc.tws.comms.dfw.io.KeyedReceiver

checkIfEmptyIsSent, finishProgress, init, isAllQueuesEmpty, isComplete, isSourcesFinished, moveMessageToSendQueue, onFinish, onMessage, progress, sendToTarget

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.comms.messaging.MessageReceiver

clean, close, onMessage

Field Detail

reduceFunction

protected ReduceFunction reduceFunction

The function that is used for the reduce operation

windowSize

protected int windowSize

Streaming messages are only kept until the window size is met. by default the window size is 1, so all messages are forwarded as they arrive.

localWindowCount

protected int localWindowCount

variable used to keep track of the current local window count. This value is always reset to 0 after it reaches the windowSize

Constructor Detail

KReduceStreamingReceiver

public KReduceStreamingReceiver()

Method Detail

offerMessage

protected boolean offerMessage(int target,
                               java.lang.Object object)

The reduce operation overrides the offer method because the reduce operation does not save all the incoming messages, it rather reduces messages with the same key and saves only the reduced values. So the messages data structure will only have a single entry for each target, key pair

Overrides:

offerMessage in class KeyedReceiver

Parameters:

target - target for which the messages are to be added

object - the message/messages to be added

Returns:

true if the message was added or false otherwise

moveMessagesToSendQueue

protected boolean moveMessagesToSendQueue(int target,
                                          java.util.Map<java.lang.Object,java.util.Queue<java.lang.Object>> messagesPerTarget)

moves all the buffered messages into the sendQueue for the given target, this method assumes that for each target that there is only one object in the queue. This is required when working with reduce operations

Overrides:

moveMessagesToSendQueue in class KeyedReceiver

Parameters:

target - target for which the move needs to be done

messagesPerTarget - messages for given target

Returns:

true if the messagesPerTarget is not empty at the end of the moving process or false otherwise