edu.iu.dsc.tws.rsched.uploaders.k8s

Class K8sUploader

java.lang.Object

edu.iu.dsc.tws.rsched.uploaders.k8s.K8sUploader

All Implemented Interfaces:

IUploader, java.lang.AutoCloseable

public class K8sUploader
extends java.lang.Object
implements IUploader

Upload the job package to either: uploader web server pods with UploaderToWebServers all job pods with DirectUploader

If there are uploader web server pods in the cluster, upload the job package to all those pods. Uploader completes after the uploading is finished. Uploader web server pods must have the label "app=twister2-uploader" This label can be set from the config with the parameter: "twister2.kubernetes.uploader.web.server.label"

If there are no uploader web servers, The job package is uploaded to each pod in the job directly. DirectUploader watches the job pods for both workers and the job master when a pod becomes Running, a thread transfers the job package to that pod. This class runs in the submitting client. It needs to run continually in the client until the job ends, to upload the job package in case of scaling up or pod failures.

Constructor Summary

Constructors

Constructor and Description
K8sUploader()

Method Summary

Modifier and Type	Method and Description
void	close() This is to for disposing or cleaning up any internal state accumulated by the uploader
boolean	complete() if uploader is threaded, this method will wait for the threaded uploading to finish
void	initialize(Config cnfg, java.lang.String jbID) Initialize the uploader with the incoming context.
boolean	undo() Undo uploading.
java.net.URI	uploadPackage(java.lang.String sourceLocation) UploadPackage will upload the job package to the given location.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

K8sUploader

public K8sUploader()

Method Detail

initialize

public void initialize(Config cnfg,
                       java.lang.String jbID)

Description copied from interface: IUploader

Initialize the uploader with the incoming context.

Specified by:

initialize in interface IUploader

uploadPackage

public java.net.URI uploadPackage(java.lang.String sourceLocation)
                           throws UploaderException

Description copied from interface: IUploader

UploadPackage will upload the job package to the given location.

Specified by:

uploadPackage in interface IUploader

Parameters:

sourceLocation - the source location with all the files

Returns:

destination URI of where the job package has been uploaded if successful, or null if failed.

Throws:

UploaderException

complete

public boolean complete()

Description copied from interface: IUploader

if uploader is threaded, this method will wait for the threaded uploading to finish

Specified by:

complete in interface IUploader

Returns:

undo

public boolean undo()

Description copied from interface: IUploader

Undo uploading. Remove uploaded files.

Specified by:

undo in interface IUploader

close

public void close()

Description copied from interface: IUploader

This is to for disposing or cleaning up any internal state accumulated by the uploader

Specified by:

close in interface IUploader

Specified by:

close in interface java.lang.AutoCloseable