edu.iu.dsc.tws.api.scheduler

Interface IUploader

All Superinterfaces:

java.lang.AutoCloseable

All Known Implementing Classes:

DirectUploader, HdfsUploader, K8sUploader, LocalFileSystemUploader, NullUploader, S3Uploader, ScpUploader, UploaderToWebServers

public interface IUploader
extends java.lang.AutoCloseable

Uploads job package to a shared location. This location must be accessible by Twister2 workers and the job master. The uploader will upload:

- job package: including user job file, config files, job definition file, and - twister2 core packages and libraries, if required

Implementation of IUploader is required to have a no argument constructor that will be called to create an instance of IUploader. initialize method must be called after the constructor

Method Summary

Modifier and Type	Method and Description
void	close() This is to for disposing or cleaning up any internal state accumulated by the uploader
default boolean	complete() if uploader is threaded, this method will wait for the threaded uploading to finish
void	initialize(Config config, java.lang.String jobID) Initialize the uploader with the incoming context.
boolean	undo() Undo uploading.
java.net.URI	uploadPackage(java.lang.String sourceLocation) UploadPackage will upload the job package to the given location.

Method Detail

initialize

void initialize(Config config,
                java.lang.String jobID)

Initialize the uploader with the incoming context.

uploadPackage

java.net.URI uploadPackage(java.lang.String sourceLocation)
                    throws UploaderException

UploadPackage will upload the job package to the given location.

Parameters:

sourceLocation - the source location with all the files

Returns:

destination URI of where the job package has been uploaded if successful, or null if failed.

Throws:

UploaderException

complete

default boolean complete()

if uploader is threaded, this method will wait for the threaded uploading to finish

Returns:

undo

boolean undo()

Undo uploading. Remove uploaded files.

close

void close()

This is to for disposing or cleaning up any internal state accumulated by the uploader

Specified by:

close in interface java.lang.AutoCloseable