Machine parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpdisp gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

machine:

type: dict

argument path: machine

batch_type:

type: str

argument path: machine/batch_type

Batch backend used to execute jobs. Option: Shell, PBS, DistributedShell, JH_UniScheduler, SlurmJobArray, LSF, OpenAPI, SGE, Torque, Slurm, Bohrium, Fugaku

local_root:

type: str | NoneType

argument path: machine/local_root

Local project root used by DPDispatcher to find task directories and local files. If submission.work_base is a relative path, it is resolved inside this directory; if submission.work_base is absolute, it is used as-is and local_root is ignored.

remote_root:

type: str | NoneType, optional

argument path: machine/remote_root

Remote root directory used by non-local contexts such as SSH. DPDispatcher creates and uses a submission-specific working directory beneath this root on the remote side. For SSHContext, this path should be absolute.

clean_asynchronously:

type: bool, optional, default: False

argument path: machine/clean_asynchronously

Clean the remote working directory asynchronously after the job finishes. Avoid enabling this while debugging, because it can remove remote artifacts before you inspect them.

retry_count:

type: int, optional, default: 3

argument path: machine/retry_count

How many times DPDispatcher will retry a failed job before raising an error.

Depending on the value of context_type, different sub args are accepted.

context_type:

type: str (flag key)

argument path: machine/context_type

possible choices: BohriumContext, LocalContext, HDFSContext, SSHContext, OpenAPIContext, LazyLocalContext

Execution context / connection type used to reach the execution environment. Option: LazyLocalContext, SSHContext, BohriumContext, OpenAPIContext, LocalContext, HDFSContext

When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):

remote_profile:

type: dict

argument path: machine[BohriumContext]/remote_profile

Configuration for Bohrium submission, including login credentials, project selection, and job-handling behavior.

email:

type: str, optional

argument path: machine[BohriumContext]/remote_profile/email

Email address used to log in to Bohrium.

password:

type: str, optional

argument path: machine[BohriumContext]/remote_profile/password

Password used together with email or phone login. If BOHR_TICKET is set, password-based login can be skipped.

phone:

type: str, optional

argument path: machine[BohriumContext]/remote_profile/phone

Phone number used to log in when email is not used.

program_id:

type: int, alias: project_id

argument path: machine[BohriumContext]/remote_profile/program_id

Program / project ID used to place uploaded jobs under the correct Bohrium project namespace.

retry_count:

type: NoneType | int, optional, default: 2

argument path: machine[BohriumContext]/remote_profile/retry_count

How many times a terminated remote job is retried on the platform side before giving up.

ignore_exit_code:

type: bool, optional, default: True

argument path: machine[BohriumContext]/remote_profile/ignore_exit_code

Whether a non-zero exit code from the remote platform is still treated as finished. If False, such jobs are marked as terminated.

keep_backup:

type: bool, optional

argument path: machine[BohriumContext]/remote_profile/keep_backup

Whether to keep uploaded/downloaded zip archives in the local backup directory after transfer.

input_data:

type: dict

argument path: machine[BohriumContext]/remote_profile/input_data

Platform-specific job configuration passed through to the Bohrium API.

When context_type is set to LocalContext (or its aliases localcontext, Local, local):

remote_profile:

type: dict, optional

argument path: machine[LocalContext]/remote_profile

Options controlling how files are staged between local_root and remote_root when both paths are on the local filesystem.

symlink:

type: bool, optional, default: True

argument path: machine[LocalContext]/remote_profile/symlink

Whether to use symbolic links instead of copying files from local_root into remote_root. Disable this when the execution side cannot access the original local path through the same filesystem view.

When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):

remote_profile:

type: dict, optional

argument path: machine[HDFSContext]/remote_profile

The information used to maintain the connection with remote machine. This field is empty for this context.

When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):

remote_profile:

type: dict

argument path: machine[SSHContext]/remote_profile

SSH connection settings for the remote machine, including authentication, timeouts, and optional proxy/jump-host behavior.

hostname:

type: str

argument path: machine[SSHContext]/remote_profile/hostname

Hostname or IP address of the SSH target machine.

username:

type: str

argument path: machine[SSHContext]/remote_profile/username

Username used to log in to the target system.

password:

type: str, optional

argument path: machine[SSHContext]/remote_profile/password

(deprecated) password of linux system. Please use SSH keys instead to improve security.

port:

type: int, optional, default: 22

argument path: machine[SSHContext]/remote_profile/port

SSH port of the target machine. Usually 22.

key_filename:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/key_filename

Path to the private key file used for SSH authentication. If left None, DPDispatcher can try discoverable keys in ~/.ssh or fall back to password-based login if configured.

passphrase:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/passphrase

Passphrase for the SSH private key, if the key is encrypted.

timeout:

type: int, optional, default: 10

argument path: machine[SSHContext]/remote_profile/timeout

Timeout in seconds for establishing the SSH connection.

totp_secret:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/totp_secret

Time-based one-time-password secret used for keyboard-interactive 2FA. It should be a base32-encoded string.

tar_compress:

type: bool, optional, default: True

argument path: machine[SSHContext]/remote_profile/tar_compress

Whether upload/download tar archives are compressed. Keeping this True usually reduces transfer size at the cost of extra CPU time.

look_for_keys:

type: bool, optional, default: True

argument path: machine[SSHContext]/remote_profile/look_for_keys

Whether to search for discoverable private key files in ~/.ssh when key_filename is not provided.

execute_command:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/execute_command

Optional command executed immediately after the SSH connection is established.

proxy_command:

type: str | NoneType, optional, default: None

argument path: machine[SSHContext]/remote_profile/proxy_command

Optional SSH ProxyCommand used to reach the target through an intermediate host or tunnel.

When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):

remote_profile:

type: dict, optional

argument path: machine[OpenAPIContext]/remote_profile

The information used to maintain the connection with remote machine. This field is empty for this context.

When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):

remote_profile:

type: dict, optional

argument path: machine[LazyLocalContext]/remote_profile

The information used to maintain the connection with remote machine. This field is empty for this context.