Machine parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpdisp gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

machine:

type: dict

argument path: machine

batch_type:

type: str

argument path: machine/batch_type

The batch job system type. Option: SlurmJobArray, Bohrium, PBS, SGE, Torque, Slurm, DistributedShell, JH_UniScheduler, OpenAPI, Shell, Fugaku, LSF

local_root:

type: str | NoneType

argument path: machine/local_root

The dir where the tasks and relating files locate. Typically the project dir.

remote_root:

type: str | NoneType, optional

argument path: machine/remote_root

The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.

clean_asynchronously:

type: bool, optional, default: False

argument path: machine/clean_asynchronously

Clean the remote directory asynchronously after the job finishes.

Depending on the value of context_type, different sub args are accepted.

context_type:

type: str (flag key)

argument path: machine/context_type

possible choices: HDFSContext, BohriumContext, LocalContext, OpenAPIContext, LazyLocalContext, SSHContext

The connection used to remote machine. Option: LocalContext, SSHContext, OpenAPIContext, BohriumContext, LazyLocalContext, HDFSContext

When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):

remote_profile:

type: dict, optional

argument path: machine[HDFSContext]/remote_profile

The information used to maintain the connection with remote machine. This field is empty for this context.

When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):

remote_profile:

type: dict

argument path: machine[BohriumContext]/remote_profile

The information used to maintain the connection with remote machine.

email:

type: str, optional

argument path: machine[BohriumContext]/remote_profile/email

Email

password:

type: str, optional

argument path: machine[BohriumContext]/remote_profile/password

Password

program_id:

type: int, alias: project_id

argument path: machine[BohriumContext]/remote_profile/program_id

Program ID

retry_count:

type: NoneType | int, optional, default: 2

argument path: machine[BohriumContext]/remote_profile/retry_count

The retry count when a job is terminated

ignore_exit_code:

type: bool, optional, default: True

argument path: machine[BohriumContext]/remote_profile/ignore_exit_code

The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,

the job state will be designated as terminated.

keep_backup:

type: bool, optional

argument path: machine[BohriumContext]/remote_profile/keep_backup

keep download and upload zip

input_data:

type: dict

argument path: machine[BohriumContext]/remote_profile/input_data

Configuration of job

When context_type is set to LocalContext (or its aliases localcontext, Local, local):

remote_profile:

type: dict, optional

argument path: machine[LocalContext]/remote_profile

The information used to maintain the local machine.

symlink:

type: bool, optional, default: True

argument path: machine[LocalContext]/remote_profile/symlink

Whether to use symbolic links to replace copy. This option should be turned off if the local directory is not accessible on the Batch system.

When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):

remote_profile:

type: dict, optional

argument path: machine[OpenAPIContext]/remote_profile

The information used to maintain the connection with remote machine. This field is empty for this context.

When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):

remote_profile:

type: dict, optional

argument path: machine[LazyLocalContext]/remote_profile

The information used to maintain the connection with remote machine. This field is empty for this context.

When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):

remote_profile:

type: dict

argument path: machine[SSHContext]/remote_profile

The information used to maintain the connection with remote machine.

hostname:

type: str

argument path: machine[SSHContext]/remote_profile/hostname

hostname or ip of ssh connection.

username:

type: str

argument path: machine[SSHContext]/remote_profile/username

username of target linux system

password:

type: str, optional

argument path: machine[SSHContext]/remote_profile/password

(deprecated) password of linux system. Please use SSH keys instead to improve security.

port:

type: int, optional, default: 22

argument path: machine[SSHContext]/remote_profile/port

ssh connection port.

key_filename:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/key_filename

key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login

passphrase:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/passphrase

passphrase of key used by ssh connection

timeout:

type: int, optional, default: 10

argument path: machine[SSHContext]/remote_profile/timeout

timeout of ssh connection

totp_secret:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/totp_secret

Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.

tar_compress:

type: bool, optional, default: True

argument path: machine[SSHContext]/remote_profile/tar_compress

The archive will be compressed in upload and download if it is True. If not, compression will be skipped.

look_for_keys:

type: bool, optional, default: True

argument path: machine[SSHContext]/remote_profile/look_for_keys

enable searching for discoverable private key files in ~/.ssh/

execute_command:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/execute_command

execute command after ssh connection is established.

proxy_command:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/proxy_command

ProxyCommand to use for SSH connection through intermediate servers.