Arguments of the submit script

fp:
type: dict
argument path: fp

The configuration for FP

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: fp/type
possible choices: vasp

the type of the fp

When type is set to vasp:

config:
type: dict, optional, default: {'command': 'vasp', 'log': 'vasp.log', 'out': 'data'}
argument path: fp[vasp]/config

Configuration of vasp runs

command:
type: str, optional, default: vasp
argument path: fp[vasp]/config/command

The command of VASP

log:
type: str, optional, default: vasp.log
argument path: fp[vasp]/config/log

The log file name of VASP

out:
type: str, optional, default: data
argument path: fp[vasp]/config/out

The output dir name of labeled data. In deepmd/npy format provided by dpdata.

task_max:
type: int, optional, default: 10
argument path: fp[vasp]/task_max

Maximum number of vasp tasks for each iteration

pp_files:
type: dict
argument path: fp[vasp]/pp_files

The pseudopotential files set by a dict, e.g. {“Al” : “path/to/the/al/pp/file”, “Mg” : “path/to/the/mg/pp/file”}

incar:
type: str
argument path: fp[vasp]/incar

The pseudopotential files set by a dict, e.g. {“Al” : “path/to/the/al/pp/file”, “Mg” : “path/to/the/mg/pp/file”}

explore:
type: dict
argument path: explore

The configuration for exploration

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: explore/type
possible choices: lmp

the type of the exploration

When type is set to lmp:

config:
type: dict, optional, default: {'command': 'lmp'}
argument path: explore[lmp]/config

Configuration of lmp exploration

command:
type: str, optional, default: lmp
argument path: explore[lmp]/config/command

The command of LAMMPS

max_numb_iter:
type: int, optional, default: 10
argument path: explore[lmp]/max_numb_iter

Maximum number of iterations per stage

conv_accuracy:
type: float, optional, default: 0.9
argument path: explore[lmp]/conv_accuracy

Convergence accuracy

fatal_at_max:
type: bool, optional, default: True
argument path: explore[lmp]/fatal_at_max

Fatal when the number of iteration per stage reaches the max_numb_iter

f_trust_lo:
type: float
argument path: explore[lmp]/f_trust_lo

Lower trust level of force model deviation

f_trust_hi:
type: float
argument path: explore[lmp]/f_trust_hi

Higher trust level of force model deviation

v_trust_lo:
type: NoneType | float, optional, default: None
argument path: explore[lmp]/v_trust_lo

Lower trust level of virial model deviation

v_trust_hi:
type: NoneType | float, optional, default: None
argument path: explore[lmp]/v_trust_hi

Higher trust level of virial model deviation

configuration_prefix:
type: NoneType | str, optional, default: None
argument path: explore[lmp]/configuration_prefix

The path prefix of lmp initial configurations

configurations:
type: list, alias: configuration
argument path: explore[lmp]/configurations

A list of initial configurations.

stages:
type: list
argument path: explore[lmp]/stages

A list of exploration stages.

train:
type: dict
argument path: train

The configuration for training

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: train/type
possible choices: dp

the type of the training

When type is set to dp:

config:
type: dict, optional, default: {'init_model_policy': 'no', 'init_model_old_ratio': 0.9, 'init_model_numb_steps': 400000, 'init_model_start_lr': 0.0001, 'init_model_start_pref_e': 0.1, 'init_model_start_pref_f': 100, 'init_model_start_pref_v': 0.0}
argument path: train[dp]/config

Number of models trained for evaluating the model deviation

init_model_policy:
type: str, optional, default: no
argument path: train[dp]/config/init_model_policy

The policy of init-model training. It can be

  • ‘no’: No init-model training. Traing from scratch.

  • ‘yes’: Do init-model training.

  • ‘old_data_larger_than:XXX’: Do init-model if the training data size of the previous model is larger than XXX. XXX is an int number.

init_model_old_ratio:
type: float, optional, default: 0.9
argument path: train[dp]/config/init_model_old_ratio

The frequency ratio of old data over new data

init_model_numb_steps:
type: int, optional, default: 400000, alias: init_model_stop_batch
argument path: train[dp]/config/init_model_numb_steps

The number of training steps when init-model

init_model_start_lr:
type: float, optional, default: 0.0001
argument path: train[dp]/config/init_model_start_lr

The start learning rate when init-model

init_model_start_pref_e:
type: float, optional, default: 0.1
argument path: train[dp]/config/init_model_start_pref_e

The start energy prefactor in loss when init-model

init_model_start_pref_f:
type: int | float, optional, default: 100
argument path: train[dp]/config/init_model_start_pref_f

The start force prefactor in loss when init-model

init_model_start_pref_v:
type: float, optional, default: 0.0
argument path: train[dp]/config/init_model_start_pref_v

The start virial prefactor in loss when init-model

numb_models:
type: int, optional, default: 4
argument path: train[dp]/numb_models

Number of models trained for evaluating the model deviation

template_script:
type: list | dict
argument path: train[dp]/template_script

Template training script. It can be a List[Dict], the length of which is the same as numb_models. Each template script in the list is used to train a model. Can be a Dict, the models share the same template training script.

inputs:
type: dict
argument path: inputs

The input parameter and artifacts for dpgen2

type_map:
type: list
argument path: inputs/type_map

The type map. e.g. [“Al”, “Mg”]. Al and Mg will have type 0 and 1, respectively.

mass_map:
type: list
argument path: inputs/mass_map

The mass map. e.g. [27., 24.]. Al and Mg will be set with mass 27. and 24. amu, respectively.

init_data_prefix:
type: NoneType | str, optional, default: None
argument path: inputs/init_data_prefix

The prefix of initial data systems

init_data_sys:
type: list
argument path: inputs/init_data_sys

The inital data systems

upload_python_packages:
type: list | NoneType | str, optional, default: None, alias: upload_python_package
argument path: upload_python_packages

Upload python package, for debug purpose

step_configs:
type: dict, optional, default: {}
argument path: step_configs

Configurations for executing dflow steps

prep_train_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/prep_train_config

Configuration for prepare train

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/prep_train_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/prep_train_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_train_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/prep_train_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/prep_train_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_train_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/prep_train_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_train_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/prep_train_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_train_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_train_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/prep_train_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/prep_train_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/prep_train_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/prep_train_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/prep_train_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/prep_train_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

run_train_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/run_train_config

Configuration for run train

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/run_train_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/run_train_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/run_train_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/run_train_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/run_train_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_train_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/run_train_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/run_train_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/run_train_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/run_train_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_train_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/run_train_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/run_train_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/run_train_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/run_train_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/run_train_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/run_train_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

prep_explore_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/prep_explore_config

Configuration for prepare exploration

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/prep_explore_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/prep_explore_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_explore_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/prep_explore_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/prep_explore_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_explore_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/prep_explore_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_explore_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/prep_explore_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_explore_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_explore_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/prep_explore_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/prep_explore_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/prep_explore_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/prep_explore_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/prep_explore_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/prep_explore_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

run_explore_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/run_explore_config

Configuration for run exploration

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/run_explore_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/run_explore_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/run_explore_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/run_explore_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/run_explore_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_explore_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/run_explore_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/run_explore_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/run_explore_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/run_explore_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_explore_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/run_explore_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/run_explore_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/run_explore_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/run_explore_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/run_explore_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/run_explore_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

prep_fp_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/prep_fp_config

Configuration for prepare fp

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/prep_fp_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/prep_fp_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_fp_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/prep_fp_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/prep_fp_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_fp_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/prep_fp_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_fp_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/prep_fp_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/prep_fp_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/prep_fp_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/prep_fp_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/prep_fp_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/prep_fp_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/prep_fp_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/prep_fp_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/prep_fp_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

run_fp_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/run_fp_config

Configuration for run fp

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/run_fp_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/run_fp_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/run_fp_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/run_fp_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/run_fp_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_fp_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/run_fp_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/run_fp_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/run_fp_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/run_fp_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/run_fp_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/run_fp_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/run_fp_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/run_fp_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/run_fp_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/run_fp_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/run_fp_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

select_confs_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/select_confs_config

Configuration for the select confs

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/select_confs_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/select_confs_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/select_confs_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/select_confs_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/select_confs_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/select_confs_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/select_confs_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/select_confs_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/select_confs_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/select_confs_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/select_confs_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/select_confs_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/select_confs_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/select_confs_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/select_confs_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/select_confs_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/select_confs_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

collect_data_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/collect_data_config

Configuration for the collect data

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/collect_data_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/collect_data_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/collect_data_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/collect_data_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/collect_data_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/collect_data_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/collect_data_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/collect_data_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/collect_data_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/collect_data_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/collect_data_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/collect_data_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/collect_data_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/collect_data_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/collect_data_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/collect_data_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/collect_data_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

cl_step_config:
type: dict, optional, default: {'template_config': {'image': 'dptechnology/dpgen2:latest', 'timeout': None, 'retry_on_transient_error': None, 'timeout_as_transient_error': False, 'envs': None}, 'continue_on_failed': False, 'continue_on_num_success': None, 'continue_on_success_ratio': None, 'parallelism': None, 'executor': None}
argument path: step_configs/cl_step_config

Configuration for the concurrent learning step

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: step_configs/cl_step_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: step_configs/cl_step_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: step_configs/cl_step_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: step_configs/cl_step_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: step_configs/cl_step_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: step_configs/cl_step_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: step_configs/cl_step_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: step_configs/cl_step_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: step_configs/cl_step_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: step_configs/cl_step_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: step_configs/cl_step_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: step_configs/cl_step_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: step_configs/cl_step_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: step_configs/cl_step_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: step_configs/cl_step_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: step_configs/cl_step_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: step_configs/cl_step_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

default_step_config:
type: dict, optional, default: {}
argument path: default_step_config

The default step configuration.

template_config:
type: dict, optional, default: {'image': 'dptechnology/dpgen2:latest'}
argument path: default_step_config/template_config

The configs passed to the PythonOPTemplate.

image:
type: str, optional, default: dptechnology/dpgen2:latest
argument path: default_step_config/template_config/image

The image to run the step.

timeout:
type: int | NoneType, optional, default: None
argument path: default_step_config/template_config/timeout

The time limit of the OP. Unit is second.

retry_on_transient_error:
type: NoneType | bool, optional, default: None
argument path: default_step_config/template_config/retry_on_transient_error

Retry the step if a TransientError is raised.

timeout_as_transient_error:
type: bool, optional, default: False
argument path: default_step_config/template_config/timeout_as_transient_error

Treat the timeout as TransientError.

envs:
type: dict | NoneType, optional, default: None
argument path: default_step_config/template_config/envs

The environmental variables.

continue_on_failed:
type: bool, optional, default: False
argument path: default_step_config/continue_on_failed

If continue the the step is failed (FatalError, TransientError, A certain number of retrial is reached…).

continue_on_num_success:
type: int | NoneType, optional, default: None
argument path: default_step_config/continue_on_num_success

Only in the sliced OP case. Continue the workflow if a certain number of the sliced jobs are successful.

continue_on_success_ratio:
type: NoneType | float, optional, default: None
argument path: default_step_config/continue_on_success_ratio

Only in the sliced OP case. Continue the workflow if a certain ratio of the sliced jobs are successful.

parallelism:
type: int | NoneType, optional, default: None
argument path: default_step_config/parallelism

The parallelism for the step

executor:
type: dict | NoneType, optional, default: None
argument path: default_step_config/executor

The executor of the step.

Depending on the value of type, different sub args are accepted.

type:
type: str (flag key)
argument path: default_step_config/executor/type
possible choices: lebesgue_v2, dispatcher

The type of the executor.

When type is set to lebesgue_v2:

extra:
type: dict, optional
argument path: default_step_config/executor[lebesgue_v2]/extra

The ‘extra’ key in the lebesgue executor. Note that we do not check if ‘the dict provided to the ‘extra’ key is valid or not.

scass_type:
type: str, optional
argument path: default_step_config/executor[lebesgue_v2]/extra/scass_type

The machine configuraiton.

program_id:
type: str, optional
argument path: default_step_config/executor[lebesgue_v2]/extra/program_id

The ID of the program.

job_type:
type: str, optional, default: container
argument path: default_step_config/executor[lebesgue_v2]/extra/job_type

The type of job.

template_cover_cmd_escape_bug:
type: bool, optional, default: True
argument path: default_step_config/executor[lebesgue_v2]/extra/template_cover_cmd_escape_bug

The key for hacking around a bug in Lebesgue.

When type is set to dispatcher:

lebesgue_context_config:
type: dict | NoneType, optional, default: None
argument path: lebesgue_context_config

Configuration passed to dflow Lebesgue context

s3_config:
type: dict | NoneType, optional, default: None
argument path: s3_config

The S3 configuration passed to dflow

dflow_config:
type: dict | NoneType, optional, default: None
argument path: dflow_config

The configuration passed to dflow