Arguments of the submit script

dflow_config:
type: dict | NoneType, optional, default: None
argument path: dflow_config

The configuration passed to dflow

dflow_s3_config:
type: dict | NoneType, optional, default: None
argument path: dflow_s3_config

The S3 configuration passed to dflow

default_step_config:
type: dict, optional, default: {}
argument path: default_step_config

The default step configuration.

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: default_step_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: default_step_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: default_step_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: default_step_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: default_step_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: default_step_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: default_step_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: default_step_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: default_step_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: default_step_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: default_step_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: default_step_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: default_step_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: default_step_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: default_step_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

bohrium_config:
type: dict | NoneType, optional, default: None
argument path: bohrium_config

Configurations for the Bohrium platform.

username:
type: str
argument path: bohrium_config/username

The username of the Bohrium platform

password:
type: str
argument path: bohrium_config/password

The password of the Bohrium platform

project_id:
type: int
argument path: bohrium_config/project_id

The project ID of the Bohrium platform

host:
type: str, optional, default: https://workflows.deepmodeling.com
argument path: bohrium_config/host

The host name of the Bohrium platform. Will overwrite dflow_config[‘host’]

k8s_api_server:
type: str, optional, default: https://workflows.deepmodeling.com
argument path: bohrium_config/k8s_api_server

The k8s server of the Bohrium platform. Will overwrite dflow_config[‘k8s_api_server’]

repo_key:
type: str, optional, default: oss-bohrium
argument path: bohrium_config/repo_key

The repo key of the Bohrium platform. Will overwrite dflow_s3_config[‘repo_key’]

storage_client:
type: str, optional, default: dflow.plugins.bohrium.TiefblueClient
argument path: bohrium_config/storage_client

The storage client of the Bohrium platform. Will overwrite dflow_s3_config[‘storage_client’]

step_configs:
type: dict, optional, default: {}
argument path: step_configs

Configurations for executing dflow steps

prep_train_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/prep_train_config

Configuration for prepare train

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/prep_train_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/prep_train_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_train_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_train_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/prep_train_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_train_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/prep_train_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_train_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_train_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/prep_train_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_train_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/prep_train_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_train_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_train_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/prep_train_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

run_train_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/run_train_config

Configuration for run train

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/run_train_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/run_train_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/run_train_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/run_train_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/run_train_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_train_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/run_train_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/run_train_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/run_train_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/run_train_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/run_train_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/run_train_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/run_train_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_train_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/run_train_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

prep_explore_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/prep_explore_config

Configuration for prepare exploration

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/prep_explore_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/prep_explore_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_explore_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_explore_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/prep_explore_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_explore_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/prep_explore_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_explore_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_explore_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/prep_explore_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_explore_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/prep_explore_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_explore_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_explore_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/prep_explore_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

run_explore_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/run_explore_config

Configuration for run exploration

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/run_explore_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/run_explore_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/run_explore_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/run_explore_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/run_explore_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_explore_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/run_explore_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/run_explore_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/run_explore_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/run_explore_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/run_explore_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/run_explore_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/run_explore_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_explore_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/run_explore_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

prep_fp_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/prep_fp_config

Configuration for prepare fp

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/prep_fp_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/prep_fp_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_fp_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_fp_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/prep_fp_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_fp_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/prep_fp_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_fp_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_fp_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/prep_fp_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_fp_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/prep_fp_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/prep_fp_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_fp_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/prep_fp_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

run_fp_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/run_fp_config

Configuration for run fp

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/run_fp_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/run_fp_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/run_fp_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/run_fp_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/run_fp_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_fp_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/run_fp_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/run_fp_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/run_fp_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/run_fp_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/run_fp_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/run_fp_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/run_fp_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_fp_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/run_fp_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

select_confs_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/select_confs_config

Configuration for the select confs

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/select_confs_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/select_confs_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/select_confs_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/select_confs_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/select_confs_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/select_confs_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/select_confs_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/select_confs_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/select_confs_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/select_confs_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/select_confs_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/select_confs_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/select_confs_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/select_confs_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/select_confs_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

collect_data_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/collect_data_config

Configuration for the collect data

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/collect_data_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/collect_data_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/collect_data_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/collect_data_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/collect_data_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/collect_data_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/collect_data_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/collect_data_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/collect_data_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/collect_data_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/collect_data_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/collect_data_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/collect_data_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/collect_data_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/collect_data_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

cl_step_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/cl_step_config

Configuration for the concurrent learning step

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/cl_step_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/cl_step_config/template_config/image

The image to run the step.

timeout:
type: NoneType | int, optional, default: None
argument path: step_configs/cl_step_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | int, optional, default: None
argument path: step_configs/cl_step_config/template_config/retry_on_transient_error

The number of retry times if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/cl_step_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/cl_step_config/template_config/envs

The environmental variables.

template_slice_config:
type: dict, optional
argument path: step_configs/cl_step_config/template_slice_config

The configs passed to the Slices.

group_size:
type: NoneType | int, optional, default: None
argument path: step_configs/cl_step_config/template_slice_config/group_size

The number of tasks running on a single node. It is efficient for a large number of short tasks.

pool_size:
type: NoneType | int, optional, default: None
argument path: step_configs/cl_step_config/template_slice_config/pool_size

The number of tasks running at the same time on one node.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/cl_step_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: NoneType | int, optional, default: None
argument path: step_configs/cl_step_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/cl_step_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: NoneType | int, optional, default: None
argument path: step_configs/cl_step_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/cl_step_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/cl_step_config/executor/type
possible choices: dispatcher

The type of the executor.

When type is set to dispatcher:

upload_python_packages:
type: str | list | NoneType, optional, default: None, alias: upload_python_package
argument path: upload_python_packages

Upload python package, for debug purpose

inputs:
type: dict
argument path: inputs

The input parameter and artifacts for dpgen2

type_map:
type: list
argument path: inputs/type_map

The type map. e.g. [“Al”, “Mg”]. Al and Mg will have type 0 and 1, respectively.

mass_map:
type: list
argument path: inputs/mass_map

The mass map. e.g. [27., 24.]. Al and Mg will be set with mass 27. and 24. amu, respectively.

init_data_prefix:
type: str | NoneType, optional, default: None
argument path: inputs/init_data_prefix

The prefix of initial data systems

mixed_type:
type: bool, optional, default: False
argument path: inputs/mixed_type

Use deepmd/npy/mixed format for storing training data.

do_finetune:
type: bool, optional, default: False
argument path: inputs/do_finetune
Finetune the pretrained model before the first iteration. If it is set to True, then an additional step, finetune-step,

which is based on a branch of “PrepRunDPTrain,” will be added before the dpgen_step. In the finetune-step, the internal flag finetune_mode is set to “finetune,” which means SuperOP “PrepRunDPTrain” is now used as the “Finetune.” In this step, we finetune the pretrained model in the train step and modify the template after training. After that, in the normal dpgen-step, the flag do_finetune is set as “train-init,” which means we use –init-frz-model to train based on models from the previous iteration. The “do_finetune” flag is set to False by default, while the internal flag finetune_mode is set to “no,” which means anything related to finetuning will not be done.

init_data_sys:
type: str | list
argument path: inputs/init_data_sys

The inital data systems

train:
type: dict
argument path: train

The configuration for training

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: train/type
possible choices: dp, dp-dist

the type of the training

When type is set to dp:

config:
type: dict, optional, default: {'init_model_policy': 'no', 'init_model_old_ratio': 0.9, 'init_model_numb_steps': 400000, 'init_model_start_lr': 0.0001, 'init_model_start_pref_e': 0.1, 'init_model_start_pref_f': 100, 'init_model_start_pref_v': 0.0}
argument path: train[dp]/config

Number of models trained for evaluating the model deviation

init_model_policy:
type: str, optional, default: no
argument path: train[dp]/config/init_model_policy

The policy of init-model training. It can be

  • ‘no’: No init-model training. Traing from scratch.

  • ‘yes’: Do init-model training.

  • ‘old_data_larger_than:XXX’: Do init-model if the training data size of the previous model is larger than XXX. XXX is an int number.

init_model_old_ratio:
type: float, optional, default: 0.9
argument path: train[dp]/config/init_model_old_ratio

The frequency ratio of old data over new data

init_model_numb_steps:
type: int, optional, default: 400000, alias: init_model_stop_batch
argument path: train[dp]/config/init_model_numb_steps

The number of training steps when init-model

init_model_start_lr:
type: float, optional, default: 0.0001
argument path: train[dp]/config/init_model_start_lr

The start learning rate when init-model

init_model_start_pref_e:
type: float, optional, default: 0.1
argument path: train[dp]/config/init_model_start_pref_e

The start energy prefactor in loss when init-model

init_model_start_pref_f:
type: int | float, optional, default: 100
argument path: train[dp]/config/init_model_start_pref_f

The start force prefactor in loss when init-model

init_model_start_pref_v:
type: float, optional, default: 0.0
argument path: train[dp]/config/init_model_start_pref_v

The start virial prefactor in loss when init-model

numb_models:
type: int, optional, default: 4
argument path: train[dp]/numb_models

Number of models trained for evaluating the model deviation

template_script:
type: str | list
argument path: train[dp]/template_script

File names of the template training script. It can be a List[str], the length of which is the same as numb_models. Each template script in the list is used to train a model. Can be a str, the models share the same template training script.

init_models_paths:
type: list | NoneType, optional, default: None, alias: training_iter0_model_path
argument path: train[dp]/init_models_paths

the paths to initial models

When type is set to dp-dist:

config:
type: dict, optional, default: {'init_model_policy': 'no', 'init_model_old_ratio': 0.9, 'init_model_numb_steps': 400000, 'init_model_start_lr': 0.0001, 'init_model_start_pref_e': 0.1, 'init_model_start_pref_f': 100, 'init_model_start_pref_v': 0.0}
argument path: train[dp-dist]/config

Configuration of training

init_model_policy:
type: str, optional, default: no
argument path: train[dp-dist]/config/init_model_policy

The policy of init-model training. It can be

  • ‘no’: No init-model training. Traing from scratch.

  • ‘yes’: Do init-model training.

  • ‘old_data_larger_than:XXX’: Do init-model if the training data size of the previous model is larger than XXX. XXX is an int number.

init_model_old_ratio:
type: float, optional, default: 0.9
argument path: train[dp-dist]/config/init_model_old_ratio

The frequency ratio of old data over new data

init_model_numb_steps:
type: int, optional, default: 400000, alias: init_model_stop_batch
argument path: train[dp-dist]/config/init_model_numb_steps

The number of training steps when init-model

init_model_start_lr:
type: float, optional, default: 0.0001
argument path: train[dp-dist]/config/init_model_start_lr

The start learning rate when init-model

init_model_start_pref_e:
type: float, optional, default: 0.1
argument path: train[dp-dist]/config/init_model_start_pref_e

The start energy prefactor in loss when init-model

init_model_start_pref_f:
type: int | float, optional, default: 100
argument path: train[dp-dist]/config/init_model_start_pref_f

The start force prefactor in loss when init-model

init_model_start_pref_v:
type: float, optional, default: 0.0
argument path: train[dp-dist]/config/init_model_start_pref_v

The start virial prefactor in loss when init-model

template_script:
type: str | list
argument path: train[dp-dist]/template_script

File names of the template training script. It can be a List[str], the length of which is the same as numb_models. Each template script in the list is used to train a model. Can be a str, the models share the same template training script.

student_model_path:
type: str
argument path: train[dp-dist]/student_model_path

The path of student model

explore:
type: dict
argument path: explore

The configuration for exploration

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: explore/type
possible choices: lmp

the type of the exploration

When type is set to lmp:

config:
type: dict, optional, default: {'command': 'lmp', 'teacher_model_path': None, 'shuffle_models': False}
argument path: explore[lmp]/config

Configuration of lmp exploration

command:
type: str, optional, default: lmp
argument path: explore[lmp]/config/command

The command of LAMMPS

teacher_model_path:
type: str | BinaryFileInput | NoneType, optional, default: None
argument path: explore[lmp]/config/teacher_model_path

The teacher model in Knowledge Distillation

shuffle_models:
type: bool, optional, default: False
argument path: explore[lmp]/config/shuffle_models

Randomly pick a model from the group of models to drive theexploration MD simulation

max_numb_iter:
type: int, optional, default: 10
argument path: explore[lmp]/max_numb_iter

Maximum number of iterations per stage

fatal_at_max:
type: bool, optional, default: True
argument path: explore[lmp]/fatal_at_max

Fatal when the number of iteration per stage reaches the max_numb_iter

output_nopbc:
type: bool, optional, default: False
argument path: explore[lmp]/output_nopbc

Remove pbc of the output configurations

convergence:
type: list | dict
argument path: explore[lmp]/convergence

The method of convergence check.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: explore[lmp]/convergence/type
possible choices: fixed-levels, fixed-levels-max-select, adaptive-lower

the type of the convergence check

When type is set to fixed-levels:

level_f_lo:
type: float
argument path: explore[lmp]/convergence[fixed-levels]/level_f_lo

The lower trust level of force model deviation

level_f_hi:
type: float
argument path: explore[lmp]/convergence[fixed-levels]/level_f_hi

The higher trust level of force model deviation

level_v_lo:
type: NoneType | float, optional, default: None
argument path: explore[lmp]/convergence[fixed-levels]/level_v_lo

The lower trust level of virial model deviation

level_v_hi:
type: NoneType | float, optional, default: None
argument path: explore[lmp]/convergence[fixed-levels]/level_v_hi

The higher trust level of virial model deviation

conv_accuracy:
type: float, optional, default: 0.9
argument path: explore[lmp]/convergence[fixed-levels]/conv_accuracy

If the ratio of accurate frames is larger than this value, the stage is converged

When type is set to fixed-levels-max-select:

level_f_lo:
type: float
argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_f_lo

The lower trust level of force model deviation

level_f_hi:
type: float
argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_f_hi

The higher trust level of force model deviation

level_v_lo:
type: NoneType | float, optional, default: None
argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_v_lo

The lower trust level of virial model deviation

level_v_hi:
type: NoneType | float, optional, default: None
argument path: explore[lmp]/convergence[fixed-levels-max-select]/level_v_hi

The higher trust level of virial model deviation

conv_accuracy:
type: float, optional, default: 0.9
argument path: explore[lmp]/convergence[fixed-levels-max-select]/conv_accuracy

If the ratio of accurate frames is larger than this value, the stage is converged

When type is set to adaptive-lower:

level_f_hi:
type: float, optional, default: 0.5
argument path: explore[lmp]/convergence[adaptive-lower]/level_f_hi

The higher trust level of force model deviation

numb_candi_f:
type: int, optional, default: 200
argument path: explore[lmp]/convergence[adaptive-lower]/numb_candi_f

The number of force frames that has a model deviation lower than level_f_hi treated as candidate.

rate_candi_f:
type: float, optional, default: 0.01
argument path: explore[lmp]/convergence[adaptive-lower]/rate_candi_f

The ratio of force frames that has a model deviation lower than level_f_hi treated as candidate.

level_v_hi:
type: NoneType | float, optional, default: None
argument path: explore[lmp]/convergence[adaptive-lower]/level_v_hi

The higher trust level of virial model deviation

numb_candi_v:
type: int, optional, default: 0
argument path: explore[lmp]/convergence[adaptive-lower]/numb_candi_v

The number of virial frames that has a model deviation lower than level_v_hi treated as candidate.

rate_candi_v:
type: float, optional, default: 0.0
argument path: explore[lmp]/convergence[adaptive-lower]/rate_candi_v

The ratio of virial frames that has a model deviation lower than level_v_hi treated as candidate.

n_checked_steps:
type: int, optional, default: 2
argument path: explore[lmp]/convergence[adaptive-lower]/n_checked_steps

The number of steps to check the convergence.

conv_tolerance:
type: float, optional, default: 0.05
argument path: explore[lmp]/convergence[adaptive-lower]/conv_tolerance

The convergence tolerance.

candi_sel_prob:
type: str, optional, default: uniform
argument path: explore[lmp]/convergence[adaptive-lower]/candi_sel_prob

The method for selecting candidates. It can be ‘uniform’: all candidates are of the same probability. ‘inv_pop_f’ or ‘inv_pop_f:nhist’: the probability is inversely propotional to the population of a histogram between leven_f_lo and level_f_hi. The number of bins in the histogram is set by nhist, which should be an integer. The default is 10.

configuration_prefix:
type: str | NoneType, optional, default: None
argument path: explore[lmp]/configuration_prefix

The path prefix of lmp initial configurations

configurations:
type: list, alias: configuration
argument path: explore[lmp]/configurations

A list of initial configurations.

This argument takes a list with each element containing the following:

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: explore[lmp]/configurations/type
possible choices: alloy, file

the type of the configuration generator

When type is set to alloy:

numb_confs:
type: int, optional, default: 1
argument path: explore[lmp]/configurations[alloy]/numb_confs

The number of configurations to generate

lattice:
type: list | tuple
argument path: explore[lmp]/configurations[alloy]/lattice

The lattice. Should be a list providing [ “lattice_type”, lattice_const ], or a list providing [ “/path/to/dpdata/system”, “fmt” ]. The two styles are distinguished by the type of the second element.

replicate:
type: list | NoneType, optional, default: None
argument path: explore[lmp]/configurations[alloy]/replicate

The number of replicates in each direction

concentration:
type: list | NoneType, optional, default: None
argument path: explore[lmp]/configurations[alloy]/concentration

The concentration of each element. If None all elements have the same concentration

cell_pert_frac:
type: float, optional, default: 0.0
argument path: explore[lmp]/configurations[alloy]/cell_pert_frac

The faction of cell perturbation

atom_pert_dist:
type: float, optional, default: 0.0
argument path: explore[lmp]/configurations[alloy]/atom_pert_dist

The distance of atomic position perturbation

When type is set to file:

files:
type: str | list
argument path: explore[lmp]/configurations[file]/files

The paths to the configuration files. widecards are supported.

prefix:
type: str | NoneType, optional, default: None
argument path: explore[lmp]/configurations[file]/prefix

The prefix of file paths.

fmt:
type: str, optional, default: auto
argument path: explore[lmp]/configurations[file]/fmt

The format (dpdata accepted formats) of the files.

remove_pbc:
type: bool, optional, default: False
argument path: explore[lmp]/configurations[file]/remove_pbc

The remove the pbc of the data. shift the coords to the center of box so it can be used with lammps.

stages:
type: list
argument path: explore[lmp]/stages

A list of exploration stages.

fp:
type: dict
argument path: fp

The configuration for FP

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: fp/type
possible choices: vasp, gaussian, deepmd

the type of the fp

When type is set to vasp:

inputs_config:
type: dict
argument path: fp[vasp]/inputs_config

Configuration for preparing vasp inputs

incar:
type: str
argument path: fp[vasp]/inputs_config/incar

The path to the template incar file

pp_files:
type: dict
argument path: fp[vasp]/inputs_config/pp_files

The pseudopotential files set by a dict, e.g. {“Al” : “path/to/the/al/pp/file”, “Mg” : “path/to/the/mg/pp/file”}

kspacing:
type: float
argument path: fp[vasp]/inputs_config/kspacing

The spacing of k-point sampling. ksapcing will overwrite the incar template

kgamma:
type: bool, optional, default: True
argument path: fp[vasp]/inputs_config/kgamma

If the k-mesh includes the gamma point. kgamma will overwrite the incar template

run_config:
type: dict
argument path: fp[vasp]/run_config

Configuration for running vasp tasks

command:
type: str, optional, default: vasp
argument path: fp[vasp]/run_config/command

The command of VASP

out:
type: str, optional, default: data
argument path: fp[vasp]/run_config/out

The output dir name of labeled data. In deepmd/npy format provided by dpdata.

log:
type: str, optional, default: fp.log
argument path: fp[vasp]/run_config/log

The log file name of VASP

task_max:
type: int, optional, default: 10
argument path: fp[vasp]/task_max

Maximum number of vasp tasks for each iteration

When type is set to gaussian:

inputs_config:
type: dict
argument path: fp[gaussian]/inputs_config

Configuration for preparing vasp inputs

keywords:
type: str | list
argument path: fp[gaussian]/inputs_config/keywords

Gaussian keywords, e.g. force b3lyp/6-31g**. If a list, run multiple steps.

multiplicity:
type: str | int, optional, default: auto
argument path: fp[gaussian]/inputs_config/multiplicity

spin multiplicity state. It can be a number. If auto, multiplicity will be detected automatically, with the following rules:

fragment_guesses=True multiplicity will +1 for each radical, and +2 for each oxygen molecule

fragment_guesses=False multiplicity will be 1 or 2, but +2 for each oxygen molecule.

charge:
type: int, optional, default: 0
argument path: fp[gaussian]/inputs_config/charge

molecule charge. Only used when charge is not provided by the system

basis_set:
type: str, optional
argument path: fp[gaussian]/inputs_config/basis_set

custom basis set

keywords_high_multiplicity:
type: str, optional
argument path: fp[gaussian]/inputs_config/keywords_high_multiplicity

keywords for points with multiple raicals. multiplicity should be auto. If not set, fallback to normal keywords

fragment_guesses:
type: bool, optional, default: False
argument path: fp[gaussian]/inputs_config/fragment_guesses

initial guess generated from fragment guesses. If True, multiplicity should be auto

nproc:
type: int, optional, default: 1
argument path: fp[gaussian]/inputs_config/nproc

Number of CPUs to use

run_config:
type: dict
argument path: fp[gaussian]/run_config

Configuration for running vasp tasks

command:
type: str, optional, default: g16
argument path: fp[gaussian]/run_config/command

The command of Gaussian

out:
type: str, optional, default: data
argument path: fp[gaussian]/run_config/out

The output dir name of labeled data. In deepmd/npy format provided by dpdata.

task_max:
type: int, optional, default: 10
argument path: fp[gaussian]/task_max

Maximum number of vasp tasks for each iteration

When type is set to deepmd:

inputs_config:
type: dict
argument path: fp[deepmd]/inputs_config

Configuration for preparing vasp inputs

run_config:
type: dict
argument path: fp[deepmd]/run_config

Configuration for running vasp tasks

teacher_model_path:
type: str | BinaryFileInput
argument path: fp[deepmd]/run_config/teacher_model_path

The path of teacher model, which can be loaded by deepmd.infer.DeepPot

out:
type: str, optional, default: data
argument path: fp[deepmd]/run_config/out

The output dir name of labeled data. In deepmd/npy format provided by dpdata.

log:
type: str, optional, default: fp.log
argument path: fp[deepmd]/run_config/log

The log file name of dp

task_max:
type: int, optional, default: 10
argument path: fp[deepmd]/task_max

Maximum number of vasp tasks for each iteration

name:
type: str, optional, default: dpgen
argument path: name

The workflow name, ‘dpgen’ for default