edu.iu.dsc.tws.comms.dfw.io

Class SourceReceiver

java.lang.Object

edu.iu.dsc.tws.comms.dfw.io.SourceReceiver

All Implemented Interfaces:

MessageReceiver

Direct Known Subclasses:

BaseGatherBatchReceiver, GatherStreamingPartialReceiver, ReduceBatchReceiver, ReduceStreamingReceiver

public abstract class SourceReceiver
extends java.lang.Object
implements MessageReceiver

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Map<java.lang.Integer,byte[]>	barriers The barriers for each target
protected int	destination The destination
protected java.util.Map<java.lang.Integer,java.lang.Boolean>	isSyncSent Weather sync messages are forwarded from the partial receivers
protected java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Queue<java.lang.Object>>>	messages Lets keep track of the messages, we need to keep track of the messages for each target and source, Map<target, map<source, Queue<messages>>
protected DataFlowOperation	operation The operations
protected int	sendPendingMax The pending max per source
protected java.util.Map<java.lang.Integer,java.lang.Integer>	sourcesOfTarget Number of sources per target
protected java.util.Map<java.lang.Integer,java.util.Set<java.lang.Integer>>	syncReceived Keep weather we have received a sync from a source
protected SyncState	syncState The sync state
protected java.util.Map<java.lang.Integer,ReceiverState>	targetStates Keep state about the targets
protected int	workerId The worker id this receiver is in

Constructor Summary

Constructors

Constructor and Description
SourceReceiver()

Method Summary

Modifier and Type	Method and Description
protected abstract boolean	aggregate(int target, boolean sync, boolean allValuesFound) Aggregate values from sources for a target, assumes every source has a value
protected boolean	allQueuesEmpty(java.util.Map<java.lang.Integer,java.util.Queue<java.lang.Object>> messagePerTarget) Check weather there is nothing left in the queues
protected boolean	allSyncsPresent(int target) Checks weather all syncs are present
void	clean() Clean any state associated with the receiver and go to initial state
void	init(Config cfg, DataFlowOperation op, java.util.Map<java.lang.Integer,java.util.List<java.lang.Integer>> expectedIds) Initialize the message receiver with tasks from which messages are expected For each sub edge in graph, for each path, gives the expected task ids
protected abstract boolean	isAllEmpty(int target) Check weather all the other information is flushed
boolean	isComplete() Weather we are complete a set of messages
protected abstract boolean	isFilledToSend(int target, boolean sync) Return true if we are filled to send
protected void	onFinish(int source)
boolean	onMessage(int source, int path, int target, int flags, java.lang.Object object) The actual message callback
protected abstract void	onSyncEvent(int target, byte[] value) This method is called when there is a sync event on the operation
boolean	progress() This method will be called by the communicationProgress
protected abstract boolean	sendSyncForward(int target) Handle the sync
protected abstract boolean	sendToTarget(int target, boolean sync) Send the values to a target

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.iu.dsc.tws.api.comms.messaging.MessageReceiver

close, onMessage

Field Detail

messages

protected java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Queue<java.lang.Object>>> messages

Lets keep track of the messages, we need to keep track of the messages for each target and source, Map<target, map<source, Queue<messages>>

workerId

protected int workerId

The worker id this receiver is in

operation

protected DataFlowOperation operation

The operations

sendPendingMax

protected int sendPendingMax

The pending max per source

destination

protected int destination

The destination

syncReceived

protected java.util.Map<java.lang.Integer,java.util.Set<java.lang.Integer>> syncReceived

Keep weather we have received a sync from a source

sourcesOfTarget

protected java.util.Map<java.lang.Integer,java.lang.Integer> sourcesOfTarget

Number of sources per target

isSyncSent

protected java.util.Map<java.lang.Integer,java.lang.Boolean> isSyncSent

Weather sync messages are forwarded from the partial receivers

targetStates

protected java.util.Map<java.lang.Integer,ReceiverState> targetStates

Keep state about the targets

barriers

protected java.util.Map<java.lang.Integer,byte[]> barriers

The barriers for each target

syncState

protected SyncState syncState

The sync state

Constructor Detail

SourceReceiver

public SourceReceiver()

Method Detail

init

public void init(Config cfg,
                 DataFlowOperation op,
                 java.util.Map<java.lang.Integer,java.util.List<java.lang.Integer>> expectedIds)

Description copied from interface: MessageReceiver

Initialize the message receiver with tasks from which messages are expected For each sub edge in graph, for each path, gives the expected task ids

target -> source tasks

Specified by:

init in interface MessageReceiver

expectedIds - expected task ids

onMessage

public boolean onMessage(int source,
                         int path,
                         int target,
                         int flags,
                         java.lang.Object object)

Description copied from interface: MessageReceiver

The actual message callback

Specified by:

onMessage in interface MessageReceiver

Parameters:

source - the source task

path - the path that is taken by the message, that is intermediate targets

target - the target of this receiver

flags - the communication flags

object - the actual message

Returns:

true if the message is accepted

progress

public boolean progress()

Description copied from interface: MessageReceiver

This method will be called by the communicationProgress

Specified by:

progress in interface MessageReceiver

Returns:

true if needs further communicationProgress

isComplete

public boolean isComplete()

Description copied from interface: MessageReceiver

Weather we are complete a set of messages

Specified by:

isComplete in interface MessageReceiver

Returns:

true if a round is complete with syncs

isAllEmpty

protected abstract boolean isAllEmpty(int target)

Check weather all the other information is flushed

Parameters:

target - target

Returns:

true if there is nothing to process

sendSyncForward

protected abstract boolean sendSyncForward(int target)

Handle the sync

Parameters:

target - target

Returns:

the needFurtherProgress

sendToTarget

protected abstract boolean sendToTarget(int target,
                                        boolean sync)

Send the values to a target

Parameters:

target - the target

Returns:

true if all the values are sent successfully

aggregate

protected abstract boolean aggregate(int target,
                                     boolean sync,
                                     boolean allValuesFound)

Aggregate values from sources for a target, assumes every source has a value

Parameters:

target - target

sync - true if all the syncs are present

Returns:

true if there are no more elements to aggregate

isFilledToSend

protected abstract boolean isFilledToSend(int target,
                                          boolean sync)

Return true if we are filled to send

Returns:

true if we are filled enough to send

allQueuesEmpty

protected boolean allQueuesEmpty(java.util.Map<java.lang.Integer,java.util.Queue<java.lang.Object>> messagePerTarget)

Check weather there is nothing left in the queues

Parameters:

messagePerTarget - the queues for the target

Returns:

true if no values left

allSyncsPresent

protected boolean allSyncsPresent(int target)

Checks weather all syncs are present

Returns:

true if all syncs are present

onFinish

protected void onFinish(int source)

clean

public void clean()

Description copied from interface: MessageReceiver

Clean any state associated with the receiver and go to initial state

Specified by:

clean in interface MessageReceiver

onSyncEvent

protected abstract void onSyncEvent(int target,
                                    byte[] value)

This method is called when there is a sync event on the operation

Parameters:

target - the target to which the sync event belong

value - the byte value, can be null