edu.iu.dsc.tws.rsched.bootstrap

Class ZKUtil

java.lang.Object

edu.iu.dsc.tws.rsched.bootstrap.ZKUtil

public final class ZKUtil
extends java.lang.Object

this class provides methods to construct znode path names for jobs and workers in addition, it provides methods to cleanup znodes on zookeeper server

Field Summary

Fields

Modifier and Type	Field and Description
static java.util.logging.Logger	LOG

Method Summary

Modifier and Type	Method and Description
static byte[]	addTwoByteArrays(byte[] byteArray1, byte[] byteArray2) add two byte arrays like appending the second one to first one, but in a new array
static CuratorFramework	connectToServer(Config config) connect to ZooKeeper server
static java.lang.String	constructBarrierPath(Config config, java.lang.String jobName) construct a distributed barrier path
static java.lang.String	constructDaiPathForBarrier(Config config, java.lang.String jobName) construct a distributed atomic integer path for barrier
static java.lang.String	constructDaiPathForWorkerID(Config config, java.lang.String jobName) construct a distributed atomic integer path for assigning worker ids
static java.lang.String	constructJobLockPath(Config config, java.lang.String jobName) construct a job distributed lock path from the given job name
static java.lang.String	constructJobMasterPath(Config config) construct a worker path from the given job path and worker network info
static java.lang.String	constructJobPath(Config config, java.lang.String jobName) construct a job path from the given job name
static java.lang.String	constructWorkerPath(java.lang.String jobPath, java.lang.String workerHostAndPort) construct a worker path from the given job path and worker network info
static PersistentNode	createPersistentEphemeralZnode(CuratorFramework client, java.lang.String path, byte[] payload) create a PersistentNode object in the given path it is ephemeral and persistent it will be deleted after the worker leaves or fails it will be persistent for occasional network problems
static PersistentNode	createPersistentZnode(CuratorFramework client, java.lang.String path, byte[] payload) create a PersistentNode object in the given path it needs to be deleted explicitly, not ephemeral it will be persistent for occasional network problems
static java.util.List<JobMasterAPI.WorkerInfo>	decodeWorkerInfos(byte[] encodedBytes) decode the given binary encoded WorkerInfo object list encoding assumed to be dones by encodeWorkerInfo method length of each WorkerInfo object is encoded before the WorkerInfo object bytes
static boolean	deleteJobZNodes(Config config, CuratorFramework client, java.lang.String jobName) delete job related znode from previous sessions
static byte[]	encodeWorkerInfo(JobMasterAPI.WorkerInfo workerInfo) encode the given WorkerInfo object as a byte array.
static int	intFromBytes(byte[] byteArray, int startIndex) construct an int from four bytes starting at the given index
static boolean	isThereAnActiveJob(java.lang.String jobName, Config config) check whether there is an active job if not, but there are znodes from previous sessions, those will be deleted
static boolean	terminateJob(java.lang.String jobName, Config config) delete all znodes related to the given jobName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final java.util.logging.Logger LOG

Method Detail

connectToServer

public static CuratorFramework connectToServer(Config config)

connect to ZooKeeper server

Parameters:

config -

Returns:

isThereAnActiveJob

public static boolean isThereAnActiveJob(java.lang.String jobName,
                                         Config config)

check whether there is an active job if not, but there are znodes from previous sessions, those will be deleted

Parameters:

jobName -

Returns:

constructJobPath

public static java.lang.String constructJobPath(Config config,
                                                java.lang.String jobName)

construct a job path from the given job name

Parameters:

jobName -

Returns:

constructDaiPathForWorkerID

public static java.lang.String constructDaiPathForWorkerID(Config config,
                                                           java.lang.String jobName)

construct a distributed atomic integer path for assigning worker ids

Parameters:

jobName -

Returns:

constructDaiPathForBarrier

public static java.lang.String constructDaiPathForBarrier(Config config,
                                                          java.lang.String jobName)

construct a distributed atomic integer path for barrier

Parameters:

jobName -

Returns:

constructBarrierPath

public static java.lang.String constructBarrierPath(Config config,
                                                    java.lang.String jobName)

construct a distributed barrier path

Parameters:

jobName -

Returns:

constructJobLockPath

public static java.lang.String constructJobLockPath(Config config,
                                                    java.lang.String jobName)

construct a job distributed lock path from the given job name

Parameters:

jobName -

Returns:

constructWorkerPath

public static java.lang.String constructWorkerPath(java.lang.String jobPath,
                                                   java.lang.String workerHostAndPort)

construct a worker path from the given job path and worker network info

Returns:

constructJobMasterPath

public static java.lang.String constructJobMasterPath(Config config)

construct a worker path from the given job path and worker network info

Returns:

deleteJobZNodes

public static boolean deleteJobZNodes(Config config,
                                      CuratorFramework client,
                                      java.lang.String jobName)

delete job related znode from previous sessions

Parameters:

jobName -

Returns:

terminateJob

public static boolean terminateJob(java.lang.String jobName,
                                   Config config)

delete all znodes related to the given jobName

Parameters:

jobName -

Returns:

createPersistentEphemeralZnode

public static PersistentNode createPersistentEphemeralZnode(CuratorFramework client,
                                                            java.lang.String path,
                                                            byte[] payload)

create a PersistentNode object in the given path it is ephemeral and persistent it will be deleted after the worker leaves or fails it will be persistent for occasional network problems

Parameters:

path -

payload -

Returns:

Throws:

java.lang.Exception

createPersistentZnode

public static PersistentNode createPersistentZnode(CuratorFramework client,
                                                   java.lang.String path,
                                                   byte[] payload)

create a PersistentNode object in the given path it needs to be deleted explicitly, not ephemeral it will be persistent for occasional network problems

Parameters:

path -

payload -

Returns:

Throws:

java.lang.Exception

decodeWorkerInfos

public static java.util.List<JobMasterAPI.WorkerInfo> decodeWorkerInfos(byte[] encodedBytes)

decode the given binary encoded WorkerInfo object list encoding assumed to be dones by encodeWorkerInfo method length of each WorkerInfo object is encoded before the WorkerInfo object bytes

Returns:

encodeWorkerInfo

public static byte[] encodeWorkerInfo(JobMasterAPI.WorkerInfo workerInfo)

encode the given WorkerInfo object as a byte array. First put the length of the byte array as a 4 byte array to the beginning resulting byte array has the length and workerInfo object after that

Returns:

addTwoByteArrays

public static byte[] addTwoByteArrays(byte[] byteArray1,
                                      byte[] byteArray2)

add two byte arrays like appending the second one to first one, but in a new array

Parameters:

byteArray1 -

byteArray2 -

Returns:

intFromBytes

public static int intFromBytes(byte[] byteArray,
                               int startIndex)

construct an int from four bytes starting at the given index

Parameters:

byteArray -

startIndex -

Returns: