edu.iu.dsc.tws.master.server

Class WorkerMonitor

java.lang.Object

edu.iu.dsc.tws.master.server.WorkerMonitor

All Implemented Interfaces:

MessageHandler

public class WorkerMonitor
extends java.lang.Object
implements MessageHandler

This class monitors upto date list of workers in a job with their status It handles worker state changes: worker join, worker running, worker completion, worker failure

It gets worker state changes either from workers directly with protocol messages or from ZooKeeper server

It handles Job Master to Dashboard communications It handles Job Master to Driver interactions

Constructor Summary

Constructors

Constructor and Description
WorkerMonitor(JobMaster jobMaster, RRServer rrServer, DashboardClient dashClient, ZKJobUpdater zkJobUpdater, JobAPI.Job job, IDriver driver, IWorkerFailureListener failureListener)

Method Summary

Modifier and Type	Method and Description
boolean	addJoinedWorkers(java.util.List<WorkerWithState> joinedWorkers) when the job master restarts, it adds already joined workers with this method.
boolean	allWorkersJoined() if all workers are in one of these states: STARTED, RESTARTED or COMPLETED.
boolean	broadcastMessage(Message message)
void	completed(int workerID) called when a worker COMPLETED the job
boolean	existWorker(int workerID) return true if there is a worker with the given workerID
void	failed(int workerID) called when a worker FAILED
void	fullyFailed(int workerID)
int	getNumberOfWorkers()
java.util.List<java.lang.Integer>	getWorkerIDs() get the list of workerIDs sorted
JobMasterAPI.WorkerInfo	getWorkerInfoForID(int id)
java.util.List<JobMasterAPI.WorkerInfo>	getWorkerInfoList() get the list of workerIDs sorted
java.util.Collection<WorkerWithState>	getWorkerList() get the list of currently registered workers
int	getWorkersListSize() get the size of workers list
JobMasterAPI.WorkerState	getWorkerState(int workerID)
void	informDriverForAllJoined() inform the driver on restarts if all workers already joined
boolean	isAllJoined()
void	onMessage(RequestID id, int workerId, Message message) Handles a specific message type
void	restarted(WorkerWithState workerWithState) if the worker is coming from failure
boolean	sendMessageToWorkerList(Message message, java.util.List<java.lang.Integer> workerList) send a protocol buffer message to a list of workers
void	setWorkerEventSender(IWorkerEventSender workerEventSender)
void	started(WorkerWithState workerWithState) new worker joins for the first time returns null if the join is successful, otherwise, it returns an explanation for the failure
void	workersScaledDown(int instancesRemoved)
void	workersScaledUp(int instancesAdded)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WorkerMonitor

public WorkerMonitor(JobMaster jobMaster,
                     RRServer rrServer,
                     DashboardClient dashClient,
                     ZKJobUpdater zkJobUpdater,
                     JobAPI.Job job,
                     IDriver driver,
                     IWorkerFailureListener failureListener)

Method Detail

setWorkerEventSender

public void setWorkerEventSender(IWorkerEventSender workerEventSender)

getNumberOfWorkers

public int getNumberOfWorkers()

getWorkersListSize

public int getWorkersListSize()

get the size of workers list

getWorkerList

public java.util.Collection<WorkerWithState> getWorkerList()

get the list of currently registered workers

getWorkerInfoForID

public JobMasterAPI.WorkerInfo getWorkerInfoForID(int id)

getWorkerState

public JobMasterAPI.WorkerState getWorkerState(int workerID)

getWorkerIDs

public java.util.List<java.lang.Integer> getWorkerIDs()

get the list of workerIDs sorted

getWorkerInfoList

public java.util.List<JobMasterAPI.WorkerInfo> getWorkerInfoList()

get the list of workerIDs sorted

existWorker

public boolean existWorker(int workerID)

return true if there is a worker with the given workerID

isAllJoined

public boolean isAllJoined()

onMessage

public void onMessage(RequestID id,
                      int workerId,
                      Message message)

Description copied from interface: MessageHandler

Handles a specific message type

Specified by:

onMessage in interface MessageHandler

Parameters:

id - request id

workerId - worker id

message - protbuf message

started

public void started(WorkerWithState workerWithState)

new worker joins for the first time returns null if the join is successful, otherwise, it returns an explanation for the failure

restarted

public void restarted(WorkerWithState workerWithState)

if the worker is coming from failure

completed

public void completed(int workerID)

called when a worker COMPLETED the job

failed

public void failed(int workerID)

called when a worker FAILED

fullyFailed

public void fullyFailed(int workerID)

workersScaledDown

public void workersScaledDown(int instancesRemoved)

workersScaledUp

public void workersScaledUp(int instancesAdded)

addJoinedWorkers

public boolean addJoinedWorkers(java.util.List<WorkerWithState> joinedWorkers)

when the job master restarts, it adds already joined workers with this method. returns true if allJoined becomes true

informDriverForAllJoined

public void informDriverForAllJoined()

inform the driver on restarts if all workers already joined

broadcastMessage

public boolean broadcastMessage(Message message)

sendMessageToWorkerList

public boolean sendMessageToWorkerList(Message message,
                                       java.util.List<java.lang.Integer> workerList)

send a protocol buffer message to a list of workers

allWorkersJoined

public boolean allWorkersJoined()

if all workers are in one of these states: STARTED, RESTARTED or COMPLETED. return true

This is used to send allWorkersJoined message We omit started but failed workers We include started but completed workers