Arguments of the submit script

dflow_config:

type: dict | NoneType, optional, default: None

argument path: dflow_config

The configuration passed to dflow

dflow_s3_config:

type: dict | NoneType, optional, default: None

argument path: dflow_s3_config

The S3 configuration passed to dflow

default_step_config:

type: dict, optional, default: {}

argument path: default_step_config

The default step configuration.

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: default_step_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: default_step_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: default_step_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: default_step_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: default_step_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: default_step_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: default_step_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: default_step_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: default_step_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: default_step_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: default_step_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: default_step_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: default_step_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: default_step_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: default_step_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

bohrium_config:

type: dict | NoneType, optional, default: None

argument path: bohrium_config

Configurations for the Bohrium platform.

username:

type: str

argument path: bohrium_config/username

The username of the Bohrium platform

password:

type: str

argument path: bohrium_config/password

The password of the Bohrium platform

project_id:

type: int

argument path: bohrium_config/project_id

The project ID of the Bohrium platform

host:

type: str, optional, default: https://workflows.deepmodeling.com

argument path: bohrium_config/host

The host name of the Bohrium platform. Will overwrite dflow_config[‘host’]

k8s_api_server:

type: str, optional, default: https://workflows.deepmodeling.com

argument path: bohrium_config/k8s_api_server

The k8s server of the Bohrium platform. Will overwrite dflow_config[‘k8s_api_server’]

repo_key:

type: str, optional, default: oss-bohrium

argument path: bohrium_config/repo_key

The repo key of the Bohrium platform. Will overwrite dflow_s3_config[‘repo_key’]

storage_client:

type: str, optional, default: dflow.plugins.bohrium.TiefblueClient

argument path: bohrium_config/storage_client

The storage client of the Bohrium platform. Will overwrite dflow_s3_config[‘storage_client’]

step_configs:

type: dict, optional, default: {}

argument path: step_configs

Configurations for executing dflow steps

prep_train_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/prep_train_config

Configuration for prepare train

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/prep_train_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/prep_train_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_train_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_train_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/prep_train_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/prep_train_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/prep_train_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_train_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_train_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/prep_train_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_train_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/prep_train_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_train_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/prep_train_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/prep_train_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

run_train_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/run_train_config

Configuration for run train

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/run_train_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/run_train_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/run_train_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/run_train_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/run_train_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/run_train_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/run_train_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/run_train_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/run_train_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/run_train_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/run_train_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/run_train_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/run_train_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/run_train_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/run_train_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

prep_explore_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/prep_explore_config

Configuration for prepare exploration

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/prep_explore_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/prep_explore_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/prep_explore_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/prep_explore_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/prep_explore_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/prep_explore_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/prep_explore_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/prep_explore_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

run_explore_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/run_explore_config

Configuration for run exploration

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/run_explore_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/run_explore_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/run_explore_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/run_explore_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/run_explore_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/run_explore_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/run_explore_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/run_explore_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/run_explore_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/run_explore_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/run_explore_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/run_explore_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/run_explore_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/run_explore_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/run_explore_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

prep_fp_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/prep_fp_config

Configuration for prepare fp

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/prep_fp_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/prep_fp_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/prep_fp_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/prep_fp_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/prep_fp_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/prep_fp_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/prep_fp_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/prep_fp_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

run_fp_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/run_fp_config

Configuration for run fp

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/run_fp_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/run_fp_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/run_fp_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/run_fp_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/run_fp_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/run_fp_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/run_fp_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/run_fp_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/run_fp_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/run_fp_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/run_fp_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/run_fp_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/run_fp_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/run_fp_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/run_fp_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

select_confs_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/select_confs_config

Configuration for the select confs

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/select_confs_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/select_confs_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/select_confs_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/select_confs_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/select_confs_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/select_confs_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/select_confs_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/select_confs_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/select_confs_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/select_confs_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/select_confs_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/select_confs_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/select_confs_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/select_confs_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/select_confs_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

collect_data_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/collect_data_config

Configuration for the collect data

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/collect_data_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/collect_data_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/collect_data_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/collect_data_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/collect_data_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/collect_data_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/collect_data_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/collect_data_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/collect_data_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/collect_data_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/collect_data_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/collect_data_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/collect_data_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/collect_data_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/collect_data_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

cl_step_config:

type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}

argument path: step_configs/cl_step_config

Configuration for the concurrent learning step

template_config:

type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}

argument path: step_configs/cl_step_config/template_config

The configs passed to the PythonOPTemplate.

image:

type: str, optional, default: dptechnology/dpgen2:latest

argument path: step_configs/cl_step_config/template_config/image

The image to run the step.

timeout:

type: int | NoneType, optional, default: None

argument path: step_configs/cl_step_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:

type: int | NoneType, optional, default: None

argument path: step_configs/cl_step_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:

type: bool, optional, default: False

argument path: step_configs/cl_step_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:

type: dict | NoneType, optional, default: None

argument path: step_configs/cl_step_config/template_config/envs

The environmental variables.

template_slice_config:

type: dict, optional

argument path: step_configs/cl_step_config/template_slice_config

The configs passed to the Slices.

group_size:

type: int | NoneType, optional, default: None

argument path: step_configs/cl_step_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:

type: int | NoneType, optional, default: None

argument path: step_configs/cl_step_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:

type: bool, optional, default: False

argument path: step_configs/cl_step_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:

type: int | NoneType, optional, default: None

argument path: step_configs/cl_step_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:

type: NoneType | float, optional, default: None

argument path: step_configs/cl_step_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:

type: int | NoneType, optional, default: None

argument path: step_configs/cl_step_config/parallelism

The parallelism for the step

executor:

type: dict | NoneType, optional, default: None

argument path: step_configs/cl_step_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: step_configs/cl_step_config/executor/type

possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

upload_python_packages:

type: list | NoneType | str, optional, default: None, alias: upload_python_package

argument path: upload_python_packages

Upload python package, for debug purpose

inputs:

type: dict

argument path: inputs

The input parameter and artifacts for dpgen2

type_map:

type: list

argument path: inputs/type_map

The type map. e.g. [“Al”, “Mg”]. Al and Mg will have type 0 and 1, respectively.

mass_map:

type: list

argument path: inputs/mass_map

The mass map. e.g. [27., 24.]. Al and Mg will be set with mass 27. and 24. amu, respectively.

init_data_prefix:

type: NoneType | str, optional, default: None

argument path: inputs/init_data_prefix

The prefix of initial data systems

mixed_type:

type: bool, optional, default: False

argument path: inputs/mixed_type

Use deepmd/npy/mixed format for storing training data.

init_data_sys:

type: list | str

argument path: inputs/init_data_sys

The inital data systems

train:

type: dict

argument path: train

The configuration for training

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: train/type

possible choices: dp, dp-dist

the type of the training

When type is set to dp:

config:

type: dict, optional, default: {'init_model_policy': 'no', 'init_model_old_ratio': 0.9, 'init_model_numb_steps': 400000, 'init_model_start_lr': 0.0001, 'init_model_start_pref_e': 0.1, 'init_model_start_pref_f': 100, 'init_model_start_pref_v': 0.0}

argument path: train[dp]/config

Number of models trained for evaluating the model deviation

init_model_policy:

type: str, optional, default: no

argument path: train[dp]/config/init_model_policy

The policy of init-model training. It can be

• ‘no’: No init-model training. Traing from scratch.

• ‘yes’: Do init-model training.

• ‘old_data_larger_than:XXX’: Do init-model if the training data size of the previous model is larger than XXX. XXX is an int number.

init_model_old_ratio:

type: float, optional, default: 0.9

argument path: train[dp]/config/init_model_old_ratio

The frequency ratio of old data over new data

init_model_numb_steps:

type: int, optional, default: 400000, alias: init_model_stop_batch

argument path: train[dp]/config/init_model_numb_steps

The number of training steps when init-model

init_model_start_lr:

type: float, optional, default: 0.0001

argument path: train[dp]/config/init_model_start_lr

The start learning rate when init-model

init_model_start_pref_e:

type: float, optional, default: 0.1

argument path: train[dp]/config/init_model_start_pref_e

The start energy prefactor in loss when init-model

init_model_start_pref_f:

type: int | float, optional, default: 100

argument path: train[dp]/config/init_model_start_pref_f

The start force prefactor in loss when init-model

init_model_start_pref_v:

type: float, optional, default: 0.0

argument path: train[dp]/config/init_model_start_pref_v

The start virial prefactor in loss when init-model

numb_models:

type: int, optional, default: 4

argument path: train[dp]/numb_models

Number of models trained for evaluating the model deviation

template_script:

type: list | str

argument path: train[dp]/template_script

File names of the template training script. It can be a List[str], the length of which is the same as numb_models. Each template script in the list is used to train a model. Can be a str, the models share the same template training script.

init_models_paths:

type: list | NoneType, optional, default: None, alias: training_iter0_model_path

argument path: train[dp]/init_models_paths

the paths to initial models

When type is set to dp-dist:

config:

type: dict, optional, default: {'init_model_policy': 'no', 'init_model_old_ratio': 0.9, 'init_model_numb_steps': 400000, 'init_model_start_lr': 0.0001, 'init_model_start_pref_e': 0.1, 'init_model_start_pref_f': 100, 'init_model_start_pref_v': 0.0}

argument path: train[dp-dist]/config

Configuration of training

init_model_policy:

type: str, optional, default: no

argument path: train[dp-dist]/config/init_model_policy

The policy of init-model training. It can be

• ‘no’: No init-model training. Traing from scratch.

• ‘yes’: Do init-model training.

• ‘old_data_larger_than:XXX’: Do init-model if the training data size of the previous model is larger than XXX. XXX is an int number.

init_model_old_ratio:

type: float, optional, default: 0.9

argument path: train[dp-dist]/config/init_model_old_ratio

The frequency ratio of old data over new data

init_model_numb_steps:

type: int, optional, default: 400000, alias: init_model_stop_batch

argument path: train[dp-dist]/config/init_model_numb_steps

The number of training steps when init-model

init_model_start_lr:

type: float, optional, default: 0.0001

argument path: train[dp-dist]/config/init_model_start_lr

The start learning rate when init-model

init_model_start_pref_e:

type: float, optional, default: 0.1

argument path: train[dp-dist]/config/init_model_start_pref_e

The start energy prefactor in loss when init-model

init_model_start_pref_f:

type: int | float, optional, default: 100

argument path: train[dp-dist]/config/init_model_start_pref_f

The start force prefactor in loss when init-model

init_model_start_pref_v:

type: float, optional, default: 0.0

argument path: train[dp-dist]/config/init_model_start_pref_v

The start virial prefactor in loss when init-model

template_script:

type: list | str

argument path: train[dp-dist]/template_script

File names of the template training script. It can be a List[str], the length of which is the same as numb_models. Each template script in the list is used to train a model. Can be a str, the models share the same template training script.

student_model_path:

type: str

argument path: train[dp-dist]/student_model_path

The path of student model

explore:

type: dict

argument path: explore

The configuration for exploration

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: explore/type

possible choices: lmp

the type of the exploration

When type is set to lmp:

config:

type: dict, optional, default: {'command': 'lmp', 'teacher_model_path': None, 'shuffle_models': False}

argument path: explore[lmp]/config

Configuration of lmp exploration

command:

type: str, optional, default: lmp

argument path: explore[lmp]/config/command

The command of LAMMPS

teacher_model_path:

type: NoneType | BinaryFileInput | str, optional, default: None

argument path: explore[lmp]/config/teacher_model_path

The teacher model in Knowledge Distillation

shuffle_models:

type: bool, optional, default: False

argument path: explore[lmp]/config/shuffle_models

Randomly pick a model from the group of models to drive theexploration MD simulation

max_numb_iter:

type: int, optional, default: 10

argument path: explore[lmp]/max_numb_iter

Maximum number of iterations per stage

fatal_at_max:

type: bool, optional, default: True

argument path: explore[lmp]/fatal_at_max

Fatal when the number of iteration per stage reaches the max_numb_iter

output_nopbc:

type: bool, optional, default: False

argument path: explore[lmp]/output_nopbc

Remove pbc of the output configurations

convergence:

type: list | dict

argument path: explore[lmp]/convergence

The method of convergence check.

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: explore[lmp]/convergence/type

possible choices: fixed-levels, fixed-levels-max-select, adaptive-lower

the type of the convergence check

When type is set to fixed-levels:

level_f_lo:

type: float

argument path: explore[lmp]/convergence[fixed-levels]/level_f_lo

The lower trust level of force model deviation

level_f_hi:

type: float

argument path: explore[lmp]/convergence[fixed-levels]/level_f_hi

The higher trust level of force model deviation

level_v_lo:

type: NoneType | float, optional, default: None

argument path: explore[lmp]/convergence[fixed-levels]/level_v_lo

The lower trust level of virial model deviation

level_v_hi:

type: NoneType | float, optional, default: None

argument path: explore[lmp]/convergence[fixed-levels]/level_v_hi

The higher trust level of virial model deviation

conv_accuracy:

type: float, optional, default: 0.9

argument path: explore[lmp]/convergence[fixed-levels]/conv_accuracy

If the ratio of accurate frames is larger than this value, the stage is converged

When type is set to fixed-levels-max-select:

level_f_lo:

type: float

argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_f_lo

The lower trust level of force model deviation

level_f_hi:

type: float

argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_f_hi

The higher trust level of force model deviation

level_v_lo:

type: NoneType | float, optional, default: None

argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_v_lo

The lower trust level of virial model deviation

level_v_hi:

type: NoneType | float, optional, default: None

argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_v_hi

The higher trust level of virial model deviation

conv_accuracy:

type: float, optional, default: 0.9

argument path: explore[lmp]/convergence[fixed-levels-max-select]/conv_accuracy

If the ratio of accurate frames is larger than this value, the stage is converged

When type is set to adaptive-lower:

level_f_hi:

type: float, optional, default: 0.5

argument path: explore[lmp]/convergence[adaptive-lower]/level_f_hi

The higher trust level of force model deviation

numb_candi_f:

type: int, optional, default: 200

argument path: explore[lmp]/convergence[adaptive-lower]/numb_candi_f

The number of force frames that has a model deviation lower than level_f_hi treated as candidate.

rate_candi_f:

type: float, optional, default: 0.01

argument path: explore[lmp]/convergence[adaptive-lower]/rate_candi_f

The ratio of force frames that has a model deviation lower than level_f_hi treated as candidate.

level_v_hi:

type: NoneType | float, optional, default: None

argument path: explore[lmp]/convergence[adaptive-lower]/level_v_hi

The higher trust level of virial model deviation

numb_candi_v:

type: int, optional, default: 0

argument path: explore[lmp]/convergence[adaptive-lower]/numb_candi_v

The number of virial frames that has a model deviation lower than level_v_hi treated as candidate.

rate_candi_v:

type: float, optional, default: 0.0

argument path: explore[lmp]/convergence[adaptive-lower]/rate_candi_v

The ratio of virial frames that has a model deviation lower than level_v_hi treated as candidate.

n_checked_steps:

type: int, optional, default: 2

argument path: explore[lmp]/convergence[adaptive-lower]/n_checked_steps

The number of steps to check the convergence.

conv_tolerance:

type: float, optional, default: 0.05

argument path: explore[lmp]/convergence[adaptive-lower]/conv_tolerance

The convergence tolerance.

configuration_prefix:

type: NoneType | str, optional, default: None

argument path: explore[lmp]/configuration_prefix

The path prefix of lmp initial configurations

configurations:

type: list, alias: configuration

argument path: explore[lmp]/configurations

A list of initial configurations.

This argument takes a list with each element containing the following:

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: explore[lmp]/configurations/type

possible choices: alloy, file

the type of the configuration generator

When type is set to alloy:

numb_confs:

type: int, optional, default: 1

argument path: explore[lmp]/configurations[alloy]/numb_confs

The number of configurations to generate

lattice:

type: list | tuple

argument path: explore[lmp]/configurations[alloy]/lattice

The lattice. Should be a list providing [ “lattice_type”, lattice_const ], or a list providing [ “/path/to/dpdata/system”, “fmt” ]. The two styles are distinguished by the type of the second element.

replicate:

type: list | NoneType, optional, default: None

argument path: explore[lmp]/configurations[alloy]/replicate

The number of replicates in each direction

concentration:

type: list | NoneType, optional, default: None

argument path: explore[lmp]/configurations[alloy]/concentration

The concentration of each element. If None all elements have the same concentration

cell_pert_frac:

type: float, optional, default: 0.0

argument path: explore[lmp]/configurations[alloy]/cell_pert_frac

The faction of cell perturbation

atom_pert_dist:

type: float, optional, default: 0.0

argument path: explore[lmp]/configurations[alloy]/atom_pert_dist

The distance of atomic position perturbation

When type is set to file:

files:

type: list | str

argument path: explore[lmp]/configurations[file]/files

The paths to the configuration files. widecards are supported.

prefix:

type: NoneType | str, optional, default: None

argument path: explore[lmp]/configurations[file]/prefix

The prefix of file paths.

fmt:

type: str, optional, default: auto

argument path: explore[lmp]/configurations[file]/fmt

The format (dpdata accepted formats) of the files.

remove_pbc:

type: bool, optional, default: False

argument path: explore[lmp]/configurations[file]/remove_pbc

The remove the pbc of the data. shift the coords to the center of box so it can be used with lammps.

stages:

type: list

argument path: explore[lmp]/stages

A list of exploration stages.

fp:

type: dict

argument path: fp

The configuration for FP

Depending on the value of type, different sub args are accepted.

type:

type: str (flag key)

argument path: fp/type

possible choices: vasp, gaussian, deepmd

the type of the fp

When type is set to vasp:

inputs_config:

type: dict

argument path: fp[vasp]/inputs_config

Configuration for preparing vasp inputs

incar:

type: str

argument path: fp[vasp]/inputs_config/incar

The path to the template incar file

pp_files:

type: dict

argument path: fp[vasp]/inputs_config/pp_files

The pseudopotential files set by a dict, e.g. {“Al” : “path/to/the/al/pp/file”, “Mg” : “path/to/the/mg/pp/file”}

kspacing:

type: float

argument path: fp[vasp]/inputs_config/kspacing

The spacing of k-point sampling. ksapcing will overwrite the incar template

kgamma:

type: bool, optional, default: True

argument path: fp[vasp]/inputs_config/kgamma

If the k-mesh includes the gamma point. kgamma will overwrite the incar template

run_config:

type: dict

argument path: fp[vasp]/run_config

Configuration for running vasp tasks

command:

type: str, optional, default: vasp

argument path: fp[vasp]/run_config/command

The command of VASP

out:

type: str, optional, default: data

argument path: fp[vasp]/run_config/out

The output dir name of labeled data. In deepmd/npy format provided by dpdata.

log:

type: str, optional, default: fp.log

argument path: fp[vasp]/run_config/log

The log file name of VASP

task_max:

type: int, optional, default: 10

argument path: fp[vasp]/task_max

Maximum number of vasp tasks for each iteration

When type is set to gaussian:

inputs_config:

type: dict

argument path: fp[gaussian]/inputs_config

Configuration for preparing vasp inputs

keywords:

type: list | str

argument path: fp[gaussian]/inputs_config/keywords

Gaussian keywords, e.g. force b3lyp/6-31g**. If a list, run multiple steps.

multiplicity:

type: int | str, optional, default: auto

argument path: fp[gaussian]/inputs_config/multiplicity

spin multiplicity state. It can be a number. If auto, multiplicity will be detected automatically, with the following rules:

fragment_guesses=True multiplicity will +1 for each radical, and +2 for each oxygen molecule

fragment_guesses=False multiplicity will be 1 or 2, but +2 for each oxygen molecule.

charge:

type: int, optional, default: 0

argument path: fp[gaussian]/inputs_config/charge

molecule charge. Only used when charge is not provided by the system

basis_set:

type: str, optional

argument path: fp[gaussian]/inputs_config/basis_set

custom basis set

keywords_high_multiplicity:

type: str, optional

argument path: fp[gaussian]/inputs_config/keywords_high_multiplicity

keywords for points with multiple raicals. multiplicity should be auto. If not set, fallback to normal keywords

fragment_guesses:

type: bool, optional, default: False

argument path: fp[gaussian]/inputs_config/fragment_guesses

initial guess generated from fragment guesses. If True, multiplicity should be auto

nproc:

type: int, optional, default: 1

argument path: fp[gaussian]/inputs_config/nproc

Number of CPUs to use

run_config:

type: dict

argument path: fp[gaussian]/run_config

Configuration for running vasp tasks

command:

type: str, optional, default: g16

argument path: fp[gaussian]/run_config/command

The command of Gaussian

out:

type: str, optional, default: data

argument path: fp[gaussian]/run_config/out

The output dir name of labeled data. In deepmd/npy format provided by dpdata.

task_max:

type: int, optional, default: 10

argument path: fp[gaussian]/task_max

Maximum number of vasp tasks for each iteration

When type is set to deepmd:

inputs_config:

type: dict

argument path: fp[deepmd]/inputs_config

Configuration for preparing vasp inputs

run_config:

type: dict

argument path: fp[deepmd]/run_config

Configuration for running vasp tasks

teacher_model_path:

type: BinaryFileInput | str

argument path: fp[deepmd]/run_config/teacher_model_path

The path of teacher model, which can be loaded by deepmd.infer.DeepPot

out:

type: str, optional, default: data

argument path: fp[deepmd]/run_config/out

The output dir name of labeled data. In deepmd/npy format provided by dpdata.

log:

type: str, optional, default: fp.log

argument path: fp[deepmd]/run_config/log

The log file name of dp

task_max:

type: int, optional, default: 10

argument path: fp[deepmd]/task_max

Maximum number of vasp tasks for each iteration