Arguments of the submit script
- dflow_config:
- type:
dict
|NoneType
, optional, default:None
argument path:dflow_config
The configuration passed to dflow
- dflow_s3_config:
- type:
dict
|NoneType
, optional, default:None
argument path:dflow_s3_config
The S3 configuration passed to dflow
- default_step_config:
- type:
dict
, optional, default:{}
argument path:default_step_config
The default step configuration.
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:default_step_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:default_step_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:default_step_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:default_step_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:default_step_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:default_step_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:default_step_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:default_step_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:default_step_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:default_step_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:default_step_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:default_step_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:default_step_config/parallelism
The parallelism for the step
- bohrium_config:
- type:
dict
|NoneType
, optional, default:None
argument path:bohrium_config
Configurations for the Bohrium platform.
- username:
- type:
str
argument path:bohrium_config/username
The username of the Bohrium platform
- password:
- type:
str
argument path:bohrium_config/password
The password of the Bohrium platform
- project_id:
- type:
int
argument path:bohrium_config/project_id
The project ID of the Bohrium platform
- host:
- type:
str
, optional, default:https://workflows.deepmodeling.com
argument path:bohrium_config/host
The host name of the Bohrium platform. Will overwrite dflow_config[‘host’]
- k8s_api_server:
- type:
str
, optional, default:https://workflows.deepmodeling.com
argument path:bohrium_config/k8s_api_server
The k8s server of the Bohrium platform. Will overwrite dflow_config[‘k8s_api_server’]
- repo_key:
- type:
str
, optional, default:oss-bohrium
argument path:bohrium_config/repo_key
The repo key of the Bohrium platform. Will overwrite dflow_s3_config[‘repo_key’]
- storage_client:
- type:
str
, optional, default:dflow.plugins.bohrium.TiefblueClient
argument path:bohrium_config/storage_client
The storage client of the Bohrium platform. Will overwrite dflow_s3_config[‘storage_client’]
- step_configs:
- type:
dict
, optional, default:{}
argument path:step_configs
Configurations for executing dflow steps
- prep_train_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/prep_train_config
Configuration for prepare train
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/prep_train_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/prep_train_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/prep_train_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/prep_train_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/prep_train_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/prep_train_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/prep_train_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/prep_train_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- run_train_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/run_train_config
Configuration for run train
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/run_train_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/run_train_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/run_train_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/run_train_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/run_train_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/run_train_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/run_train_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/run_train_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- prep_explore_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/prep_explore_config
Configuration for prepare exploration
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/prep_explore_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/prep_explore_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/prep_explore_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/prep_explore_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/prep_explore_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/prep_explore_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/prep_explore_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/prep_explore_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- run_explore_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/run_explore_config
Configuration for run exploration
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/run_explore_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/run_explore_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/run_explore_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/run_explore_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/run_explore_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/run_explore_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/run_explore_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/run_explore_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- prep_fp_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/prep_fp_config
Configuration for prepare fp
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/prep_fp_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/prep_fp_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/prep_fp_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/prep_fp_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/prep_fp_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/prep_fp_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/prep_fp_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/prep_fp_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- run_fp_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/run_fp_config
Configuration for run fp
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/run_fp_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/run_fp_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/run_fp_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/run_fp_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/run_fp_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/run_fp_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/run_fp_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/run_fp_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- select_confs_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/select_confs_config
Configuration for the select confs
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/select_confs_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/select_confs_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/select_confs_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/select_confs_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/select_confs_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/select_confs_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/select_confs_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/select_confs_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- collect_data_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/collect_data_config
Configuration for the collect data
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/collect_data_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/collect_data_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/collect_data_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/collect_data_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/collect_data_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/collect_data_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/collect_data_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/collect_data_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- cl_step_config:
- type:
dict
, optional, default:{'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path:step_configs/cl_step_config
Configuration for the concurrent learning step
- template_config:
- type:
dict
, optional, default:{'image': 'dptechnology/dpgen2:latest'}
argument path:step_configs/cl_step_config/template_config
The configs passed to the PythonOPTemplate.
- image:
- type:
str
, optional, default:dptechnology/dpgen2:latest
argument path:step_configs/cl_step_config/template_config/image
The image to run the step.
- timeout:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/template_config/timeout
The time limit of the OP. Unit is second.
- retry_on_transient_error:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/template_config/retry_on_transient_error
The number of retry times if a TransientError is raised.
- timeout_as_transient_error:
- type:
bool
, optional, default:False
argument path:step_configs/cl_step_config/template_config/timeout_as_transient_error
Treat the timeout as TransientError.
- envs:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/template_config/envs
The environmental variables.
- template_slice_config:
- type:
dict
, optionalargument path:step_configs/cl_step_config/template_slice_config
The configs passed to the Slices.
- group_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/template_slice_config/group_size
The number of tasks running on a single node. It is efficient for a large number of short tasks.
- pool_size:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/template_slice_config/pool_size
The number of tasks running at the same time on one node.
- continue_on_failed:
- type:
bool
, optional, default:False
argument path:step_configs/cl_step_config/continue_on_failed
If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).
- continue_on_num_success:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/continue_on_num_success
Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.
- continue_on_success_ratio:
- type:
NoneType
|float
, optional, default:None
argument path:step_configs/cl_step_config/continue_on_success_ratio
Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.
- parallelism:
- type:
int
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/parallelism
The parallelism for the step
- executor:
- type:
dict
|NoneType
, optional, default:None
argument path:step_configs/cl_step_config/executor
The executor of the step.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:step_configs/cl_step_config/executor/type
possible choices:dispatcher
The type of the executor.
When type is set to
dispatcher
:
- upload_python_packages:
- type:
list
|NoneType
|str
, optional, default:None
, alias: upload_python_packageargument path:upload_python_packages
Upload python package, for debug purpose
- inputs:
- type:
dict
argument path:inputs
The input parameter and artifacts for dpgen2
- type_map:
- type:
list
argument path:inputs/type_map
The type map. e.g. [“Al”, “Mg”]. Al and Mg will have type 0 and 1, respectively.
- mass_map:
- type:
list
argument path:inputs/mass_map
The mass map. e.g. [27., 24.]. Al and Mg will be set with mass 27. and 24. amu, respectively.
- init_data_prefix:
- type:
NoneType
|str
, optional, default:None
argument path:inputs/init_data_prefix
The prefix of initial data systems
- mixed_type:
- type:
bool
, optional, default:False
argument path:inputs/mixed_type
Use deepmd/npy/mixed format for storing training data.
- init_data_sys:
- type:
list
|str
argument path:inputs/init_data_sys
The inital data systems
- train:
- type:
dict
argument path:train
The configuration for training
Depending on the value of type, different sub args are accepted.
- type:
When type is set to
dp
:- config:
- type:
dict
, optional, default:{'init_model_policy': 'no', 'init_model_old_ratio': 0.9, 'init_model_numb_steps': 400000, 'init_model_start_lr': 0.0001, 'init_model_start_pref_e': 0.1, 'init_model_start_pref_f': 100, 'init_model_start_pref_v': 0.0}
argument path:train[dp]/config
Number of models trained for evaluating the model deviation
- init_model_policy:
- type:
str
, optional, default:no
argument path:train[dp]/config/init_model_policy
The policy of init-model training. It can be
‘no’: No init-model training. Traing from scratch.
‘yes’: Do init-model training.
‘old_data_larger_than:XXX’: Do init-model if the training data size of the previous model is larger than XXX. XXX is an int number.
- init_model_old_ratio:
- type:
float
, optional, default:0.9
argument path:train[dp]/config/init_model_old_ratio
The frequency ratio of old data over new data
- init_model_numb_steps:
- type:
int
, optional, default:400000
, alias: init_model_stop_batchargument path:train[dp]/config/init_model_numb_steps
The number of training steps when init-model
- init_model_start_lr:
- type:
float
, optional, default:0.0001
argument path:train[dp]/config/init_model_start_lr
The start learning rate when init-model
- init_model_start_pref_e:
- type:
float
, optional, default:0.1
argument path:train[dp]/config/init_model_start_pref_e
The start energy prefactor in loss when init-model
- init_model_start_pref_f:
- type:
int
|float
, optional, default:100
argument path:train[dp]/config/init_model_start_pref_f
The start force prefactor in loss when init-model
- init_model_start_pref_v:
- type:
float
, optional, default:0.0
argument path:train[dp]/config/init_model_start_pref_v
The start virial prefactor in loss when init-model
- numb_models:
- type:
int
, optional, default:4
argument path:train[dp]/numb_models
Number of models trained for evaluating the model deviation
- template_script:
- type:
list
|str
argument path:train[dp]/template_script
File names of the template training script. It can be a List[str], the length of which is the same as numb_models. Each template script in the list is used to train a model. Can be a str, the models share the same template training script.
- init_models_paths:
- type:
list
|NoneType
, optional, default:None
, alias: training_iter0_model_pathargument path:train[dp]/init_models_paths
the paths to initial models
When type is set to
dp-dist
:- config:
- type:
dict
, optional, default:{'init_model_policy': 'no', 'init_model_old_ratio': 0.9, 'init_model_numb_steps': 400000, 'init_model_start_lr': 0.0001, 'init_model_start_pref_e': 0.1, 'init_model_start_pref_f': 100, 'init_model_start_pref_v': 0.0}
argument path:train[dp-dist]/config
Configuration of training
- init_model_policy:
- type:
str
, optional, default:no
argument path:train[dp-dist]/config/init_model_policy
The policy of init-model training. It can be
‘no’: No init-model training. Traing from scratch.
‘yes’: Do init-model training.
‘old_data_larger_than:XXX’: Do init-model if the training data size of the previous model is larger than XXX. XXX is an int number.
- init_model_old_ratio:
- type:
float
, optional, default:0.9
argument path:train[dp-dist]/config/init_model_old_ratio
The frequency ratio of old data over new data
- init_model_numb_steps:
- type:
int
, optional, default:400000
, alias: init_model_stop_batchargument path:train[dp-dist]/config/init_model_numb_steps
The number of training steps when init-model
- init_model_start_lr:
- type:
float
, optional, default:0.0001
argument path:train[dp-dist]/config/init_model_start_lr
The start learning rate when init-model
- init_model_start_pref_e:
- type:
float
, optional, default:0.1
argument path:train[dp-dist]/config/init_model_start_pref_e
The start energy prefactor in loss when init-model
- init_model_start_pref_f:
- type:
int
|float
, optional, default:100
argument path:train[dp-dist]/config/init_model_start_pref_f
The start force prefactor in loss when init-model
- init_model_start_pref_v:
- type:
float
, optional, default:0.0
argument path:train[dp-dist]/config/init_model_start_pref_v
The start virial prefactor in loss when init-model
- template_script:
- type:
list
|str
argument path:train[dp-dist]/template_script
File names of the template training script. It can be a List[str], the length of which is the same as numb_models. Each template script in the list is used to train a model. Can be a str, the models share the same template training script.
- student_model_path:
- type:
str
argument path:train[dp-dist]/student_model_path
The path of student model
- explore:
- type:
dict
argument path:explore
The configuration for exploration
Depending on the value of type, different sub args are accepted.
- type:
the type of the exploration
When type is set to
lmp
:- config:
- type:
dict
, optional, default:{'command': 'lmp', 'teacher_model_path': None, 'shuffle_models': False}
argument path:explore[lmp]/config
Configuration of lmp exploration
- command:
- type:
str
, optional, default:lmp
argument path:explore[lmp]/config/command
The command of LAMMPS
- teacher_model_path:
- type:
NoneType
|BinaryFileInput
|str
, optional, default:None
argument path:explore[lmp]/config/teacher_model_path
The teacher model in Knowledge Distillation
- shuffle_models:
- type:
bool
, optional, default:False
argument path:explore[lmp]/config/shuffle_models
Randomly pick a model from the group of models to drive theexploration MD simulation
- max_numb_iter:
- type:
int
, optional, default:10
argument path:explore[lmp]/max_numb_iter
Maximum number of iterations per stage
- fatal_at_max:
- type:
bool
, optional, default:True
argument path:explore[lmp]/fatal_at_max
Fatal when the number of iteration per stage reaches the max_numb_iter
- output_nopbc:
- type:
bool
, optional, default:False
argument path:explore[lmp]/output_nopbc
Remove pbc of the output configurations
- convergence:
- type:
list
|dict
argument path:explore[lmp]/convergence
The method of convergence check.
Depending on the value of type, different sub args are accepted.
- type:
- type:
str
(flag key)argument path:explore[lmp]/convergence/type
the type of the convergence check
When type is set to
fixed-levels
:- level_f_lo:
- type:
float
argument path:explore[lmp]/convergence[fixed-levels]/level_f_lo
The lower trust level of force model deviation
- level_f_hi:
- type:
float
argument path:explore[lmp]/convergence[fixed-levels]/level_f_hi
The higher trust level of force model deviation
- level_v_lo:
- type:
NoneType
|float
, optional, default:None
argument path:explore[lmp]/convergence[fixed-levels]/level_v_lo
The lower trust level of virial model deviation
- level_v_hi:
- type:
NoneType
|float
, optional, default:None
argument path:explore[lmp]/convergence[fixed-levels]/level_v_hi
The higher trust level of virial model deviation
- conv_accuracy:
- type:
float
, optional, default:0.9
argument path:explore[lmp]/convergence[fixed-levels]/conv_accuracy
If the ratio of accurate frames is larger than this value, the stage is converged
When type is set to
fixed-levels-max-select
:- level_f_lo:
- type:
float
argument path:explore[lmp]/convergence[fixed-levels-max-select]/level_f_lo
The lower trust level of force model deviation
- level_f_hi:
- type:
float
argument path:explore[lmp]/convergence[fixed-levels-max-select]/level_f_hi
The higher trust level of force model deviation
- level_v_lo:
- type:
NoneType
|float
, optional, default:None
argument path:explore[lmp]/convergence[fixed-levels-max-select]/level_v_lo
The lower trust level of virial model deviation
- level_v_hi:
- type:
NoneType
|float
, optional, default:None
argument path:explore[lmp]/convergence[fixed-levels-max-select]/level_v_hi
The higher trust level of virial model deviation
- conv_accuracy:
- type:
float
, optional, default:0.9
argument path:explore[lmp]/convergence[fixed-levels-max-select]/conv_accuracy
If the ratio of accurate frames is larger than this value, the stage is converged
When type is set to
adaptive-lower
:- level_f_hi:
- type:
float
, optional, default:0.5
argument path:explore[lmp]/convergence[adaptive-lower]/level_f_hi
The higher trust level of force model deviation
- numb_candi_f:
- type:
int
, optional, default:200
argument path:explore[lmp]/convergence[adaptive-lower]/numb_candi_f
The number of force frames that has a model deviation lower than level_f_hi treated as candidate.
- rate_candi_f:
- type:
float
, optional, default:0.01
argument path:explore[lmp]/convergence[adaptive-lower]/rate_candi_f
The ratio of force frames that has a model deviation lower than level_f_hi treated as candidate.
- level_v_hi:
- type:
NoneType
|float
, optional, default:None
argument path:explore[lmp]/convergence[adaptive-lower]/level_v_hi
The higher trust level of virial model deviation
- numb_candi_v:
- type:
int
, optional, default:0
argument path:explore[lmp]/convergence[adaptive-lower]/numb_candi_v
The number of virial frames that has a model deviation lower than level_v_hi treated as candidate.
- rate_candi_v:
- type:
float
, optional, default:0.0
argument path:explore[lmp]/convergence[adaptive-lower]/rate_candi_v
The ratio of virial frames that has a model deviation lower than level_v_hi treated as candidate.
- n_checked_steps:
- type:
int
, optional, default:2
argument path:explore[lmp]/convergence[adaptive-lower]/n_checked_steps
The number of steps to check the convergence.
- conv_tolerance:
- type:
float
, optional, default:0.05
argument path:explore[lmp]/convergence[adaptive-lower]/conv_tolerance
The convergence tolerance.
- configuration_prefix:
- type:
NoneType
|str
, optional, default:None
argument path:explore[lmp]/configuration_prefix
The path prefix of lmp initial configurations
- configurations:
- type:
list
, alias: configurationargument path:explore[lmp]/configurations
A list of initial configurations.
This argument takes a list with each element containing the following:
Depending on the value of type, different sub args are accepted.
- type:
When type is set to
alloy
:- numb_confs:
- type:
int
, optional, default:1
argument path:explore[lmp]/configurations[alloy]/numb_confs
The number of configurations to generate
- lattice:
- type:
list
|tuple
argument path:explore[lmp]/configurations[alloy]/lattice
The lattice. Should be a list providing [ “lattice_type”, lattice_const ], or a list providing [ “/path/to/dpdata/system”, “fmt” ]. The two styles are distinguished by the type of the second element.
- replicate:
- type:
list
|NoneType
, optional, default:None
argument path:explore[lmp]/configurations[alloy]/replicate
The number of replicates in each direction
- concentration:
- type:
list
|NoneType
, optional, default:None
argument path:explore[lmp]/configurations[alloy]/concentration
The concentration of each element. If None all elements have the same concentration
- cell_pert_frac:
- type:
float
, optional, default:0.0
argument path:explore[lmp]/configurations[alloy]/cell_pert_frac
The faction of cell perturbation
- atom_pert_dist:
- type:
float
, optional, default:0.0
argument path:explore[lmp]/configurations[alloy]/atom_pert_dist
The distance of atomic position perturbation
When type is set to
file
:- files:
- type:
list
|str
argument path:explore[lmp]/configurations[file]/files
The paths to the configuration files. widecards are supported.
- prefix:
- type:
NoneType
|str
, optional, default:None
argument path:explore[lmp]/configurations[file]/prefix
The prefix of file paths.
- fmt:
- type:
str
, optional, default:auto
argument path:explore[lmp]/configurations[file]/fmt
The format (dpdata accepted formats) of the files.
- remove_pbc:
- type:
bool
, optional, default:False
argument path:explore[lmp]/configurations[file]/remove_pbc
The remove the pbc of the data. shift the coords to the center of box so it can be used with lammps.
- stages:
- type:
list
argument path:explore[lmp]/stages
A list of exploration stages.
- fp:
- type:
dict
argument path:fp
The configuration for FP
Depending on the value of type, different sub args are accepted.
- type:
When type is set to
vasp
:- inputs_config:
- type:
dict
argument path:fp[vasp]/inputs_config
Configuration for preparing vasp inputs
- incar:
- type:
str
argument path:fp[vasp]/inputs_config/incar
The path to the template incar file
- pp_files:
- type:
dict
argument path:fp[vasp]/inputs_config/pp_files
The pseudopotential files set by a dict, e.g. {“Al” : “path/to/the/al/pp/file”, “Mg” : “path/to/the/mg/pp/file”}
- kspacing:
- type:
float
argument path:fp[vasp]/inputs_config/kspacing
The spacing of k-point sampling. ksapcing will overwrite the incar template
- kgamma:
- type:
bool
, optional, default:True
argument path:fp[vasp]/inputs_config/kgamma
If the k-mesh includes the gamma point. kgamma will overwrite the incar template
- run_config:
- type:
dict
argument path:fp[vasp]/run_config
Configuration for running vasp tasks
- command:
- type:
str
, optional, default:vasp
argument path:fp[vasp]/run_config/command
The command of VASP
- out:
- type:
str
, optional, default:data
argument path:fp[vasp]/run_config/out
The output dir name of labeled data. In deepmd/npy format provided by dpdata.
- log:
- type:
str
, optional, default:fp.log
argument path:fp[vasp]/run_config/log
The log file name of VASP
- task_max:
- type:
int
, optional, default:10
argument path:fp[vasp]/task_max
Maximum number of vasp tasks for each iteration
When type is set to
gaussian
:- inputs_config:
- type:
dict
argument path:fp[gaussian]/inputs_config
Configuration for preparing vasp inputs
- keywords:
- type:
list
|str
argument path:fp[gaussian]/inputs_config/keywords
Gaussian keywords, e.g. force b3lyp/6-31g**. If a list, run multiple steps.
- multiplicity:
- type:
int
|str
, optional, default:auto
argument path:fp[gaussian]/inputs_config/multiplicity
spin multiplicity state. It can be a number. If auto, multiplicity will be detected automatically, with the following rules:
fragment_guesses=True multiplicity will +1 for each radical, and +2 for each oxygen molecule
fragment_guesses=False multiplicity will be 1 or 2, but +2 for each oxygen molecule.
- charge:
- type:
int
, optional, default:0
argument path:fp[gaussian]/inputs_config/charge
molecule charge. Only used when charge is not provided by the system
- basis_set:
- type:
str
, optionalargument path:fp[gaussian]/inputs_config/basis_set
custom basis set
- keywords_high_multiplicity:
- type:
str
, optionalargument path:fp[gaussian]/inputs_config/keywords_high_multiplicity
keywords for points with multiple raicals. multiplicity should be auto. If not set, fallback to normal keywords
- fragment_guesses:
- type:
bool
, optional, default:False
argument path:fp[gaussian]/inputs_config/fragment_guesses
initial guess generated from fragment guesses. If True, multiplicity should be auto
- nproc:
- type:
int
, optional, default:1
argument path:fp[gaussian]/inputs_config/nproc
Number of CPUs to use
- run_config:
- type:
dict
argument path:fp[gaussian]/run_config
Configuration for running vasp tasks
- command:
- type:
str
, optional, default:g16
argument path:fp[gaussian]/run_config/command
The command of Gaussian
- out:
- type:
str
, optional, default:data
argument path:fp[gaussian]/run_config/out
The output dir name of labeled data. In deepmd/npy format provided by dpdata.
- task_max:
- type:
int
, optional, default:10
argument path:fp[gaussian]/task_max
Maximum number of vasp tasks for each iteration
When type is set to
deepmd
:- inputs_config:
- type:
dict
argument path:fp[deepmd]/inputs_config
Configuration for preparing vasp inputs
- run_config:
- type:
dict
argument path:fp[deepmd]/run_config
Configuration for running vasp tasks
- teacher_model_path:
- type:
BinaryFileInput
|str
argument path:fp[deepmd]/run_config/teacher_model_path
The path of teacher model, which can be loaded by deepmd.infer.DeepPot
- out:
- type:
str
, optional, default:data
argument path:fp[deepmd]/run_config/out
The output dir name of labeled data. In deepmd/npy format provided by dpdata.
- log:
- type:
str
, optional, default:fp.log
argument path:fp[deepmd]/run_config/log
The log file name of dp
- task_max:
- type:
int
, optional, default:10
argument path:fp[deepmd]/task_max
Maximum number of vasp tasks for each iteration