Supported contexts

Context is the way to connect to the remote server. One needs to set context_type to one of the following values:

LazyLocal

context_type: LazyLocal

LazyLocal directly runs jobs in the local server and local directory.

Since bash -l is used in the shebang line of the submission scripts, the login shell startup files will be executed, potentially overriding the current environmental variables. Therefore, it’s advisable to explicitly set the environmental variables using envs or source_list.

Local

context_type: Local

Local runs jobs in the local server, but in a different directory. Files will be copied to the remote directory before jobs start and copied back after jobs finish.

Since bash -l is used in the shebang line of the submission scripts, the login shell startup files will be executed, potentially overriding the current environmental variables. Therefore, it’s advisable to explicitly set the environmental variables using envs or source_list.

SSH

context_type: SSH

SSH runs jobs in a remote server. Files will be copied to the remote directory via SSH channels before jobs start and copied back after jobs finish. To use SSH, one needs to provide necessary parameters in remote_profile, such as username and hostname.

It’s suggested to generate SSH keys and transfer the public key to the remote server in advance, which is more secure than password authentication.

Note that SSH context is non-login, so bash_profile files will not be executed outside the submission script.

Bohrium

context_type: Bohrium

Bohrium is the cloud platform for scientific computing. Read Bohrium documentation for details. To use Bohrium, one needs to provide necessary parameters in remote_profile.

HDFS

context_type: HDFS

The Hadoop Distributed File System (HDFS) is a distributed file system. Read Support DPDispatcher on Yarn for details.

OpenAPI

context_type: OpenAPI

OpenAPI is a new way to submit jobs to Bohrium. It using AccessKey instead of username and password. Read Bohrium documentation for details. To use OpenAPI, one needs to provide necessary parameters in remote_profile.