Twister2

Twister2

  • Getting Started
  • Docs
  • Tutorial
  • AI
  • Examples
  • Contribute
  • Download
  • Configurations
  • Java Docs
  • GitHub
  • Blog

›Compiling

Compiling

  • Overview
  • Linux
  • MacOS
  • Maven Artifacts

App Development

  • API Overview
  • Developing Applications
  • Streaming Jobs
  • Batch Jobs

APIs

  • Worker API
  • Data API
  • Compute API
  • Operator API
  • Windowing API
  • Storm API
  • Apache Beam
  • Python API

Deployment

  • Job Submit
  • Standalone
  • Docker
  • Kubernetes
  • Minikube
  • Mesos
  • Nomad
  • Slurm
  • Dashboard
  • Logging
  • Configurations

Concepts

  • Overview
  • Architecture
  • Operators
  • Task System
  • Data Access

Resources

  • Publications

Linux

Prerequisites

Twister2 build needs several software installed on your system.

  1. Operating System
    • Twister2 is tested and known to work on,
      • Red Hat Enterprise Linux Server release 7
      • Ubuntu 16.10 and Ubuntu 18.10
  2. Java
    • Download Oracle JDK 8 from http://www.oracle.com/technetwork/java/javase/downloads/index.html
    • Extract the archive to a folder named jdk1.8.0
    • Set the following environment variables.
JAVA_HOME=
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH
which java
:exclamation: After installing java, you may follow below two steps to automatically configure your environment and build twister2. Currently this works only on Debian distributions with sudo access. Skip steps upto Twister2 Distribution if this is applicable.
git clone https://github.com/DSC-SPIDAL/twister2.git
cd twister2
./build_linux.sh
  1. Install the required tools
sudo apt-get install g++ git build-essential automake cmake libtool-bin zip libunwind-setjmp0-dev zlib1g-dev unzip pkg-config python-setuptools -y libnuma-dev libc-dev
sudo apt-get install  python3-dev python3-pip

Now install wheel

sudo pip install wheel
  1. Installing maven and configure it as follows :
  wget https://archive.apache.org/dist/maven/maven-3/3.6.2/binaries/apache-maven-3.6.2-bin.tar.gz

Extract this to a directory called maven configure the environmental variables

  MAVEN_HOME=
  PATH=$MAVEN_HOME/bin:$PATH
  export MAVEN_HOME PATH
  1. Install bazel 1.1.0

    wget https://github.com/bazelbuild/bazel/releases/download/1.1.0/bazel-1.1.0-installer-linux-x86_64.sh
    chmod +x bazel-1.1.0-installer-linux-x86_64.sh
    ./bazel-1.1.0-installer-linux-x86_64.sh --user
    

    Make sure to add the bazel bin to PATH. You can add this line to ~\.bashrc file.

    export PATH=$PATH:~/bin
    

Compiling Twister2

Now lets get a clone of the source code.

git clone https://github.com/DSC-SPIDAL/twister2.git

You can compile the Twister2 distribution by using the bazel target as below.

cd twister2
bazel build --config=ubuntu scripts/package:tarpkgs

This will build twister2 distribution in the file

bazel-bin/scripts/package/twister2-0.4.0.tar.gz

If you would like to compile the twister2 without building the distribution packages use the command

bazel build --config=ubuntu twister2/...

For compiling a specific target such as communications

bazel build --config=ubuntu twister2/comms/src/java:comms-java

Twister2 Distribution

After you've build the Twister2 distribution, you can extract it and use it to submit jobs.

cd bazel-bin/scripts/package/
tar -xvf twister2-0.4.0.tar.gz
← OverviewMacOS →
  • Prerequisites
  • Compiling Twister2
  • Twister2 Distribution
Twister2
Docs
Getting Started (Quickstart)Guides (Programming Guides)
Community
Stack OverflowProject Chat
More
BlogGitHubStar
Copyright © 2020 Indiana University