edu.iu.dsc.tws.comms.dfw.io.gather.keyed

Class DKGatherBatchFinalReceiver

java.lang.Object

edu.iu.dsc.tws.comms.dfw.io.KeyedReceiver

edu.iu.dsc.tws.comms.dfw.io.gather.keyed.DKGatherBatchFinalReceiver

All Implemented Interfaces:

MessageReceiver

public class DKGatherBatchFinalReceiver
extends KeyedReceiver

Disk based Final receiver for keyed gather

Field Summary

Fields inherited from class edu.iu.dsc.tws.comms.dfw.io.KeyedReceiver

batchDone, dataFlowOperation, destination, executor, finishedSources, isEmptySent, isFinalBatchReceiver, keyLimit, limitPerKey, messages, representSource, representSourceSet, sendQueue, thisSources

Constructor Summary

Constructors

Constructor and Description
DKGatherBatchFinalReceiver(BulkReceiver receiver, boolean srt, int limitPerKey, java.lang.String shuffleDir, java.util.Comparator com, boolean groupByKey)

Method Summary

Modifier and Type	Method and Description
protected boolean	checkIfEmptyIsSent(int target) checks if the Empty message was sent for this target and sends it if not sent and possible to send.
protected boolean	finishProgress(boolean needsFurtherProgress, int target) Performs the final steps of the progress method in the receiver.
void	init(Config cfg, DataFlowOperation op, java.util.Map<java.lang.Integer,java.util.List<java.lang.Integer>> expectedIds) Initialize the message receiver with tasks from which messages are expected For each sub edge in graph, for each path, gives the expected task ids
protected boolean	sendToTarget(boolean needsFurtherProgress, boolean sourcesFinished, int target, java.util.Queue<java.lang.Object> targetSendQueue) Called from the progress method to perform the communication calls to send the queued messages Since this is the disk based gather this method will save the values to disk

Methods inherited from class edu.iu.dsc.tws.comms.dfw.io.KeyedReceiver

isAllQueuesEmpty, isComplete, isSourcesFinished, moveMessagesToSendQueue, moveMessageToSendQueue, offerMessage, onFinish, onMessage, progress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.comms.messaging.MessageReceiver

clean, close, onMessage

Constructor Detail

DKGatherBatchFinalReceiver

public DKGatherBatchFinalReceiver(BulkReceiver receiver,
                                  boolean srt,
                                  int limitPerKey,
                                  java.lang.String shuffleDir,
                                  java.util.Comparator com,
                                  boolean groupByKey)

Method Detail

init

public void init(Config cfg,
                 DataFlowOperation op,
                 java.util.Map<java.lang.Integer,java.util.List<java.lang.Integer>> expectedIds)

Description copied from interface: MessageReceiver

Initialize the message receiver with tasks from which messages are expected For each sub edge in graph, for each path, gives the expected task ids

target -> source tasks

Specified by:

init in interface MessageReceiver

Overrides:

init in class KeyedReceiver

expectedIds - expected task ids

sendToTarget

protected boolean sendToTarget(boolean needsFurtherProgress,
                               boolean sourcesFinished,
                               int target,
                               java.util.Queue<java.lang.Object> targetSendQueue)

Called from the progress method to perform the communication calls to send the queued messages Since this is the disk based gather this method will save the values to disk

Overrides:

sendToTarget in class KeyedReceiver

Parameters:

needsFurtherProgress - current state of needsFurtherProgress value

sourcesFinished - specifies if the sources have completed

target - the target(which is a source in this instance) from which the messages are sent

targetSendQueue - the data structure that contains all the message data

Returns:

true if further progress is needed or false otherwise

finishProgress

protected boolean finishProgress(boolean needsFurtherProgress,
                                 int target)

Performs the final steps of the progress method in the receiver. Since this is the disk based gather the finish method will signal the Shuffle that the provided target is completed.

Overrides:

finishProgress in class KeyedReceiver

Parameters:

needsFurtherProgress - current state of needsFurtherProgress value

target - the target(which is a source in this instance) from which the messages are sent

Returns:

true if further progress is needed or false otherwise

checkIfEmptyIsSent

protected boolean checkIfEmptyIsSent(int target)

checks if the Empty message was sent for this target and sends it if not sent and possible to send. Since this is the final receiver we do not need any functionality to send empty message so the method is overwritten to simply return true

Overrides:

checkIfEmptyIsSent in class KeyedReceiver

Parameters:

target - target for which the check is done

Returns:

false if Empty is sent