Context is the way to connect to the remote server. One needs to set
context_type to one of the following values:
LazyLocal directly runs jobs in the local server and local directory.
Local runs jobs in the local server, but in a different directory. Files will be copied to the remote directory before jobs start and copied back after jobs finish.
SSH runs jobs in a remote server. Files will be copied to the remote directory via SSH channels before jobs start and copied back after jobs finish. To use SSH, one needs to provide necessary parameters in
remote_profile, such as
It’s suggested to generate SSH keys and transfer the public key to the remote server in advance, which is more secure than password authentication.
SSH context is non-login, so
bash_profile files will not be executed.
Bohrium is the cloud platform for scientific computing. Read Bohrium documentation for details. To use Bohrium, one needs to provide necessary parameters in
The Hadoop Distributed File System (HDFS) is a distributed file system. Read Support DPDispatcher on Yarn for details.