DPGEN’s documentation

Overview

About DP-GEN

Citations

DP-GEN (Deep Generator) is a software written in Python, delicately designed to generate a deep learning based model of interatomic potential energy and force field. DP-GEN is dependent on DeepMD-kit. With highly scalable interface with common softwares for molecular simulation, DP-GEN is capable to automatically prepare scripts and maintain job queues on HPC machines (High Performance Cluster) and analyze results.

If you use this software in any publication, please cite:

Yuzhi Zhang, Haidi Wang, Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang, and Weinan E, DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models, Computer Physics Communications, 2020, 107206.

Highlighted features

Accurate and efficient: DP-GEN is capable to sample more than tens of million structures and select only a few for first principles calculation. DP-GEN will finally obtain a uniformly accurate model.
User-friendly and automatic: Users may install and run DP-GEN easily. Once succusefully running, DP-GEN can dispatch and handle all jobs on HPCs, and thus there’s no need for any personal effort.
Highly scalable: With modularized code structures, users and developers can easily extend DP-GEN for their most relevant needs. DP-GEN currently supports for HPC systems (Slurm, PBS, LSF and cloud machines ), Deep Potential interface with DeePMD-kit, MD interface with LAMMPS, Gromacs and ab-initio calculation interface with VASP, PWSCF, CP2K, SIESTA and Gaussian, Abacus, PWMAT, etc . We’re sincerely welcome and embraced to users’ contributions, with more possibilities and cases to use DP-GEN.

Download and install

DP-GEN only supports Python 3.9 and above. You can use one of the following methods to install DP-GEN:

Install via pip: pip install dpgen
Install via conda: `conda install -c conda-forge dpgen``
Install from source code: git clone https://github.com/deepmodeling/dpgen && pip install ./dpgen

To test if the installation is successful, you may execute

dpgen -h

Use DP-GEN

A quick-start on using DPGEN can be found here. You can follow the Handson tutorial, it is friendly to new users.

Case Studies

Practical-Guidelines-for-DP

Before starting a new Deep Potential (DP) project, we suggest people (especially those who are newbies) read the following context first to get some insights into what tools we can use, what kinds of risks and difficulties we may meet, and how we can advance a new DP project smoothly.

Convergence-Test

to ensure the data quality, the reliability of the final model, as well as the feasibility of the project, a convergence test should be done first.

Gas-phase

In this tutorial, we will take the simulation of methane combustion as an example and introduce the procedure of DP-based MD simulation.

Mg-Y_alloy

We will briefly analyze the candidate configurational space of a metallic system by taking Mg-based Mg-Y binary alloy as an example. The task is divided into steps during the DP-GEN process.

Transfer-learning

This tutorial will introduce how to implement potential energy surface (PES) transfer-learning by using the DP-GEN software. In DP-GEN (version > 0.8.0), the “simplify” module is designed for this purpose.

License

The project dpgen is licensed under GNU LGPLv3.0

Command line interface

dpgen is a convenient script that uses DeepGenerator to prepare initial data, drive DeepMDkit and analyze results. This script works based on several sub-commands with their own options. To see the options for the sub-commands, type “dpgen sub-command -h”.

usage: dpgen [-h]
             {init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db,gui}
             ...

Sub-commands

init_surf

Generating initial data for surface systems.

dpgen init_surf [-h] PARAM [MACHINE]

Positional Arguments

PARAM: parameter file, json/yaml format
MACHINE: machine file, json/yaml format

init_bulk

Generating initial data for bulk systems.

dpgen init_bulk [-h] PARAM [MACHINE]

Positional Arguments

PARAM: parameter file, json/yaml format
MACHINE: machine file, json/yaml format

auto_gen_param

auto gen param.json

dpgen auto_gen_param [-h] PARAM

Positional Arguments

PARAM: parameter file, json/yaml format

init_reaction

Generating initial data for reactive systems.

dpgen init_reaction [-h] PARAM [MACHINE]

Positional Arguments

PARAM: parameter file, json/yaml format
MACHINE: machine file, json/yaml format

run

Main process of Deep Potential Generator.

dpgen run [-h] [-d] PARAM MACHINE

Positional Arguments

PARAM: parameter file, json/yaml format
MACHINE: machine file, json/yaml format

Named Arguments

-d, --debug

log debug info

Default: False

run/report

Report the systems and the thermodynamic conditions of the labeled frames.

dpgen run/report [-h] [-s] [-i] [-t] [-p PARAM] [-v] JOB_DIR

Positional Arguments

JOB_DIR: the directory of the DP-GEN job,

Named Arguments

-s, --stat-sys

count the labeled frames for each system

Default: False

-i, --stat-iter

print the iteration candidate,failed,accurate count and fp calculation,success and fail count

Default: False

-t, --stat-time

print the iteration time, warning!! assume model_devi parallel cores == 1

Default: False

-p, --param

the json file provides DP-GEN paramters, should be located in JOB_DIR

Default: “param.json”

-v, --verbose

being loud

Default: False

collect

Collect data.

dpgen collect [-h] [-p PARAMETER] [-v] [-m] [-s] JOB_DIR OUTPUT

Positional Arguments

JOB_DIR: the directory of the DP-GEN job
OUTPUT: the output directory of data

Named Arguments

-p, --parameter

the json file provides DP-GEN paramters, should be located in JOB_DIR

Default: “param.json”

-v, --verbose

print number of data in each system

Default: False

-m, --merge

merge the systems with the same chemical formula

Default: False

-s, --shuffle

shuffle the data systems

Default: False

simplify

Simplify data.

dpgen simplify [-h] [-d] PARAM MACHINE

Positional Arguments

PARAM: parameter file, json/yaml format
MACHINE: machine file, json/yaml format

Named Arguments

-d, --debug

log debug info

Default: False

autotest

Auto-test for Deep Potential.

dpgen autotest [-h] [-d] TASK PARAM [MACHINE]

Positional Arguments

TASK: task can be make, run or post
PARAM: parameter file, json/yaml format
MACHINE: machine file, json/yaml format

Named Arguments

-d, --debug

log debug info

Default: False

db

Collecting data from DP-GEN.

dpgen db [-h] PARAM

Positional Arguments

PARAM: parameter file, json format

gui

Serve DP-GUI.

dpgen gui [-h] [-p PORT] [--bind_all]

Named Arguments

-p, --port

The port to serve DP-GUI on.

Default: 6042

--bind_all

Serve on all public interfaces. This will expose your DP-GUI instance to the network on both IPv4 and IPv6 (where available).

Default: False

Code Structure

Let’s look at the home page of DP-GEN. https://github.com/deepmodeling/dpgen

├── build
├── CITATION.cff
├── conda
├── dist
├── doc
├── dpgen
├── dpgen.egg-info
├── examples
├── LICENSE
├── README.md
├── requirements.txt
├── setup.py
└── tests

tests : unittest tools for developers.
examples: templates for PARAM and MACHINE files for different software, versions and tasks. For details of the parameters in PARAM, you can refer to TASK parameters chapters in this document. If you are confused about how to set up a JSON file, you can also use dpgui

Most of the code related to DP-GEN functions is in the dpgen directory. Open the dpgen directory, and we can see

├── arginfo.py
├── auto_test
├── collect
├── data
├── database
├── _date.py
├── dispatcher
├── generator
├── __init__.py
├── main.py
├── __pycache__
├── remote
├── simplify
├── tools
├── util.py
└── _version.py

auto_test corresponds to dpgen autotest, for undertaking materials property analysis.
collect corresponds to dpgen collect.
data corresponds to dpgen init_bulk, dpgen init_surf and dpgen init_reaction, for preparing initial data of bulk and surf systems.
database is the source code for collecting data generated by DP-GEN and interface with database.
simplify corresponds to dpgen simplify.
remote and dispatcher : source code for automatically submiting scripts,maintaining job queues and collecting results. Notice this part hase been integrated into dpdispatcher generator is the core part of DP-GEN. It’s for main process of deep generator. Let’s open this folder.

├── arginfo.py
├── ch4
├── __init__.py
├── lib
└── run.py

run.py is the core of DP-GEN, corresponding to dpgen run. We can find make_train, run_train, … post_fp, and other steps related functions here.

Run

Overview of the Run process

The run process contains a series of successive iterations, undertaken in order such as heating the system to certain temperatures. Each iteration is composed of three steps: exploration, labeling, and training. Accordingly, there are three sub-folders: 00.train, 01.model_devi, and 02.fp in each iteration.

00.train: DP-GEN will train several (default 4) models based on initial and generated data. The only difference between these models is the random seed for neural network initialization.

01.model_devi : represent for model-deviation. DP-GEN will use models obtained from 00.train to run Molecular Dynamics(default LAMMPS). Larger deviation for structure properties (default is the force of atoms) means less accuracy of the models. Using this criterion, a few structures will be selected and put into the next stage 02.fp for more accurate calculation based on First Principles.

02.fp : Selected structures will be calculated by first-principles methods(default VASP). DP-GEN will obtain some new data and put them together with initial data and data generated in previous iterations. After that, new training will be set up and DP-GEN will enter the next iteration!

In the run process of the DP-GEN, we need to specify the basic information about the system, the initial data, and details of the training, exploration, and labeling tasks. In addition, we need to specify the software, machine environment, and computing resource and enable the process of job generation, submission, query, and collection automatically. We can perform the run process as we expect by specifying the keywords in param.json and machine.json, and they will be introduced in detail in the following sections.

Here, we give a general description of the run process. We can execute the run process of DP-GEN easily by:

dpgen run param.json machine.json

The following files or folders will be created and upgraded by codes：

iter.00000x contains the main results that DP-GEN generates in the first iteration.
record.dpgen records the current stage of the run process.
dpgen.log includes time and iteration information.

When the first iteration is completed, the folder structure of iter.000000 is like this:

$ ls iter.000000
00.train 01.model_devi 02.fp

In folder iter.000000/ 00.train:

Folder 00x contains the input and output files of the DeePMD-kit, in which a model is trained.
graph.00x.pb is the model DeePMD-kit generates. The only difference between these models is the random seed for neural network initialization.

In folder iter.000000/ 01.model_devi：

Folder confs contains the initial configurations for LAMMPS MD converted from POSCAR you set in sys_configs of param.json.
Folder task.000.00000x contains the input and output files of the LAMMPS. In folder task.000.00000x, file model_devi.out records the model deviation of concerned labels, energy and force in MD. It serves as the criterion for selecting which structures and doing first-principle calculations.

In folder iter.000000/ 02.fp：

candidate.shuffle.000.out records which structures will be selected from last step 01.model_devi. There are always far more candidates than the maximum you expect to calculate at one time. In this condition, DP-GEN will randomly choose up to fp_task_max structures and form the folder task.*.
rest_accurate.shuffle.000.out records the other structures where our model is accurate (max_devi_f is less than model_devi_f_trust_lo, no need to calculate any more),
rest_failed.shuffled.000.out records the other structures where our model is too inaccurate (larger than model_devi_f_trust_hi, there may be some error).
data.000: After first-principle calculations, DP-GEN will collect these data and change them into the format DeePMD-kit needs. In the next iteration’s 00.train, these data will be trained together as well as the initial data.

DP-GEN identifies the stage of the run process by a record file, record.dpgen, which will be created and upgraded by codes. Each line contains two numbers: the first is the index of iteration, and the second, ranging from 0 to 9, records which stage in each iteration is currently running.

Index of iterations	Stage in eachiteration	Process
0	0	make_train
0	1	run_train
0	2	post_train
0	3	make_model_devi
0	4	run_model_devi
0	5	post_model_devi
0	6	make_fp
0	7	run_fp
0	8	post_fp

0,1,2 correspond to make_train, run_train, post_train. DP-GEN will write scripts in make_train, run the task by specific machine in run_train and collect result in post_train. The records for model_devi and fp stage follow similar rules.

If the process of DP-GEN stops for some reasons, DP-GEN will automatically recover the main process by record.dpgen. You may also change it manually for your purpose, such as removing the last iterations and recovering from one checkpoint. When re-running dpgen, the process will start from the stage that the last line record.

Example-of-param.json

We have provided different examples of param.json in dpgen/examples/run/. In this section, we give a description of the param.json, taking dpgen/examples/run/dp2.x-lammps-vasp/param_CH4_deepmd-kit-2.0.1.json as an example. This is a param.json for a gas-phase methane molecule. Here, DeePMD-kit (v2.x), LAMMPS and VASP codes are used for training, exploration and labeling respectively.

basics

The basics related keys in param.json are given as follows

  "type_map": [
    "H",
    "C"
  ],
  "mass_map": [
    1,
    12
  ],

The basics related keys specify the basic information about the system. type_map gives the atom types, i.e. “H” and “C”. mass_map gives the standard atom weights, i.e. “1” and “12”.

data

The data related keys in param.json are given as follows

  "init_data_prefix": "....../init/",
  "init_data_sys": [
    "CH4.POSCAR.01x01x01/02.md/sys-0004-0001/deepmd"
  ],

  "sys_configs_prefix": "....../init/",
  "sys_configs": [
    [
      "CH4.POSCAR.01x01x01/01.scale_pert/sys-0004-0001/scale*/00000*/POSCAR"
    ],
    [
      "CH4.POSCAR.01x01x01/01.scale_pert/sys-0004-0001/scale*/00001*/POSCAR"
    ]
  ],

The data related keys specify the init data for training initial DP models and structures used for model_devi calculations. init_data_prefix and init_data_sys specify the location of the init data. sys_configs_prefix and sys_configs specify the location of the structures.

Here, the init data is provided at “…… /init/CH4.POSCAR.01x01x01/02.md/sys-0004-0001/deepmd”. These structures are divided into two groups and provided at “……/init/CH4.POSCAR.01x01x01/01.scale_pert/sys-0004-0001/scale*/00000*/POSCAR” and “……/init/CH4.POSCAR.01x01x01/01.scale_pert/sys-0004-0001/scale*/00001*/POSCAR”.

training

The training related keys in param.json are given as follows

  "numb_models": 4,
  "default_training_param": {
  },

The training related keys specify the details of training tasks. numb_models specifies the number of models to be trained. “default_training_param” specifies the training parameters for deepmd-kit.

Here, 4 DP models will be trained in 00.train. A detailed explanation of training parameters can be found in DeePMD-kit’s documentation (https://docs.deepmodeling.com/projects/deepmd/en/master/).

exploration

The exploration related keys in param.json are given as follows

  "model_devi_dt": 0.002,
  "model_devi_skip": 0,
  "model_devi_f_trust_lo": 0.05,
  "model_devi_f_trust_hi": 0.15,
  "model_devi_clean_traj": true,
  "model_devi_jobs": [
    {
      "sys_idx": [
        0
      ],
      "temps": [
        100
      ],
      "press": [
        1.0
      ],
      "trj_freq": 10,
      "nsteps": 300,
      "ensemble": "nvt",
      "_idx": "00"
    },
    {
      "sys_idx": [
        1
      ],
      "temps": [
        100
      ],
      "press": [
        1.0
      ],
      "trj_freq": 10,
      "nsteps": 3000,
      "ensemble": "nvt",
      "_idx": "01"
    }
  ],

The exploration related keys specify the details of exploration tasks. model_devi_dt specifies timestep for MD simulation. model_devi_skip specifies the number of structures skipped for saving in each MD. model_devi_f_trust_lo and model_devi_f_trust_hi specify the lower and upper bound of model_devi of forces for the selection. model_devi_clean_traj specifies whether to clean traj folders in MD. If type of model_devi_clean_traj is boolean type then it denote whether to clean traj folders in MD since they are too large. In model_devi_jobs, sys_idx specifies the group of structures used for model_devi calculations, temps specifies the temperature (K) in MD, press specifies the pressure (Bar) in MD, trj_freq specifies the frequency of trajectory saved in MD, nsteps specifies the running steps of MD, ensemble specifies the ensemble used in MD, and “_idx” specifies the index of iteration.

Here, MD simulations are performed at the temperature of 100 K and the pressure of 1.0 Bar with an integrator time of 2 fs under the nvt ensemble. Two iterations are set in model_devi_jobs. MD simulations are run for 300 and 3000 time steps with the first and second groups of structures in sys_configs in 00 and 01 iterations. We choose to save all structures generated in MD simulations and have set trj_freq as 10, so 30 and 300 structures are saved in 00 and 01 iterations. If the “max_devi_f” of saved structure falls between 0.05 and 0.15, DP-GEN will treat the structure as a candidate. We choose to clean traj folders in MD since they are too large. If you want to save the most recent n iterations of traj folders, you can set model_devi_clean_traj to be an integer.

labeling

The labeling related keys in param.json are given as follows

  "fp_style": "vasp",
  "shuffle_poscar": false,
  "fp_task_max": 20,
  "fp_task_min": 1,
  "fp_pp_path": "....../methane/",
  "fp_pp_files": [
    "POTCAR"
  ],
  "fp_incar": "....../INCAR_methane"

The labeling related keys specify the details of labeling tasks. fp_style specifies software for First Principles. fp_task_max and fp_task_min specify the minimum and maximum of structures to be calculated in 02.fp of each iteration. fp_pp_path and fp_pp_files specify the location of the psuedo-potential file to be used for 02.fp. run_jdata[fp_style=vasp]/fp_incar specifies input file for VASP. INCAR must specify KSPACING and KGAMMA.

Here, a minimum of 1 and a maximum of 20 structures will be labeled using the VASP code with the INCAR provided at “……/INCAR_methane” and POTCAR provided at “……/methane/POTCAR” in each iteration. Note that the order of elements in POTCAR should correspond to the order in type_map.

All the keys of the DP-GEN are explained in detail in the section Parameters.

Example of machine.json

DPDispatcher Update Note

DPDispatcher has updated and the api of machine.json is changed. DP-GEN will use the new DPDispatcher if the value of key api_version in machine.json is equal to or large than 1.0. And for now, DPDispatcher is maintained on a separate repo (https://github.com/deepmodeling/dpdispatcher). Please check the documents (https://deepmd.readthedocs.io/projects/dpdispatcher/en/latest/) for more information about the new DPDispatcher.

DP-GEN will use the old DPDispatcher if the key api_version is not specified in machine.json or the api_version is smaller than 1.0. This gurantees that the old machine.json still works.

New DPDispatcher

Each iteration in the run process of DP-GEN is composed of three steps: exploration, labeling, and training. Accordingly, machine.json is composed of three parts: train, model_devi, and fp. Each part is a list of dicts. Each dict can be considered as an independent environment for calculation.

In this section, we will show you how to perform train task at a local workstation, model_devi task at a local Slurm cluster, and fp task at a remote PBS cluster using the new DPDispatcher. For each task, three types of keys are needed:

Command: provides the command used to execute each step.
Machine: specifies the machine environment (local workstation, local or remote cluster, or cloud server).
Resources: specify the number of groups, nodes, CPU, and GPU; enable the virtual environment.

Performing train task at a local workstation

In this example, we perform the train task on a local workstation.

"train":
    {
      "command": "dp",
      "machine": {
        "batch_type": "Shell",
        "context_type": "local",
        "local_root": "./",
        "remote_root": "/home/user1234/work_path"
      },
      "resources": {
        "number_node": 1,
        "cpu_per_node": 4,
        "gpu_per_node": 1,
        "group_size": 1,
        "source_list": ["/home/user1234/deepmd.env"]
      }
    },

The command for the train task in the DeePMD-kit is “dp”.

In machine parameters, batch_type specifies the type of job scheduling system. If there is no job scheduling system, we can use the “Shell” to perform the task. context_type specifies the method of data transfer, and “local” means copying and moving data via local file storage systems (e.g. cp, mv, etc.). In DP-GEN, the paths of all tasks are automatically located and set by the software, and therefore local_root is always set to “./”. The input file for each task will be sent to the remote_root and the task will be performed there, so we need to make sure that the path exists.

In the resources parameter, number_node, cpu_per_node, and gpu_per_node specify the number of nodes, the number of CPUs, and the number of GPUs required for a task respectively. group_size, which needs to be highlighted, specifies how many tasks will be packed into a group. In the training tasks, we need to train 4 models. If we only have one GPU, we can set the group_size to 4. If group_size is set to 1, 4 models will be trained on one GPU at the same time, as there is no job scheduling system. Finally, the environment variables can be activated by source_list. In this example, “source /home/user1234/deepmd.env” is executed before “dp” to load the environment variables necessary to perform the training task.

Perform model_devi task at a local Slurm cluster

In this example, we perform the model_devi task at a local Slurm workstation.

"model_devi":
    {
      "command": "lmp",
      "machine": {
       "context_type": "local",
        "batch_type": "Slurm",
        "local_root": "./",
        "remote_root": "/home/user1234/work_path"
      },
      "resources": {
        "number_node": 1,
        "cpu_per_node": 4,
        "gpu_per_node": 1,
        "queue_name": "QueueGPU",
        "custom_flags" : ["#SBATCH --mem=32G"],
        "group_size": 10,
        "source_list": ["/home/user1234/lammps.env"]
      }
    }

The command for the model_devi task in the LAMMPS is “lmp”.

In the machine parameter, we specify the type of job scheduling system by changing the batch_type to “Slurm”.

In the resources parameter, we specify the name of the queue to which the task is submitted by adding queue_name. We can add additional lines to the calculation script via the custom_flags. In the model_devi steps, there are frequently many short tasks, so we usually pack multiple tasks (e.g. 10) into a group for submission. Other parameters are similar to that of the local workstation.

Perform fp task in a remote PBS cluster

In this example, we perform the fp task at a remote PBS cluster that can be accessed via SSH.

"fp":
    {
      "command": "mpirun -n 32 vasp_std",
      "machine": {
       "context_type": "SSHContext",
        "batch_type": "PBS",
        "local_root": "./",
        "remote_root": "/home/user1234/work_path",
        "remote_profile": {
          "hostname": "39.xxx.xx.xx",
          "username": "user1234"
         }
      },
      "resources": {
        "number_node": 1,
        "cpu_per_node": 32,
        "gpu_per_node": 0,
        "queue_name": "QueueCPU",
        "group_size": 5,
        "source_list": ["/home/user1234/vasp.env"]
      }
    }

VASP code is used for fp task and mpi is used for parallel computing, so “mpirun -n 32” is added to specify the number of parallel threads.

In the machine parameter, context_type is modified to “SSHContext” and batch_type is modified to “PBS”. It is worth noting that remote_root should be set to an accessible path on the remote PBS cluster. remote_profile is added to specify the information used to connect the remote cluster, including hostname, username, port, etc.

In the resources parameter, we set gpu_per_node to 0 since it is cost-effective to use the CPU for VASP calculations.

Explicit descriptions of keys in machine.json will be given in the following section.

dpgen run param parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

run_jdata:

type: dict
argument path: run_jdata
param.json file
type_map:
type: list[str]
argument path: run_jdata/type_map
Atom types. Reminder: The elements in param.json, type.raw and data.lmp(when using lammps) should be in the same order.
mass_map:
type: str | list[float], optional, default: auto
argument path: run_jdata/mass_map
Standard atomic weights (default: “auto”). if one want to use isotopes, or non-standard element names, chemical symbols, or atomic number in the type_map list, please customize the mass_map list instead of using “auto”.
use_ele_temp:
type: int, optional, default: 0
argument path: run_jdata/use_ele_temp
Currently only support fp_style vasp.
0: no electron temperature.
1: eletron temperature as frame parameter.
2: electron temperature as atom parameter.
init_data_prefix:
type: str, optional
argument path: run_jdata/init_data_prefix
Prefix of initial data directories.
init_data_sys:
type: list[str]
argument path: run_jdata/init_data_sys
Paths of initial data. The path can be either a system diretory containing NumPy files or an HDF5 file. You may use either absolute or relative path here. Systems will be detected recursively in the directories or the HDF5 file.
sys_format:
type: str, optional, default: vasp/poscar
argument path: run_jdata/sys_format
Format of sys_configs.
init_batch_size:
type: str | list[typing.Union[int, str]], optional
argument path: run_jdata/init_batch_size
Each number is the batch_size of corresponding system for training in init_data_sys. One recommended rule for setting the sys_batch_size and init_batch_size is that batch_size mutiply number of atoms ot the stucture should be larger than 32. If set to auto, batch size will be 32 divided by number of atoms. This argument will not override the mixed batch size in default_training_param.
sys_configs_prefix:
type: str, optional
argument path: run_jdata/sys_configs_prefix
Prefix of sys_configs.
sys_configs:
type: list[list[str]]
argument path: run_jdata/sys_configs
2D list. Containing directories of structures to be explored in iterations for each system. Wildcard characters are supported here.
sys_batch_size:
type: list[typing.Union[int, str]], optional
argument path: run_jdata/sys_batch_size
Each number is the batch_size for training of corresponding system in sys_configs. If set to auto, batch size will be 32 divided by number of atoms. This argument will not override the mixed batch size in default_training_param.
train_backend:
type: str, optional, default: tensorflow
argument path: run_jdata/train_backend
The backend of the training. Currently only support tensorflow and pytorch.
numb_models:
type: int
argument path: run_jdata/numb_models
Number of models to be trained in 00.train. 4 is recommend.
training_iter0_model_path:
type: list[str], optional
argument path: run_jdata/training_iter0_model_path
The model used to init the first iter training. Number of element should be equal to numb_models.
training_init_model:
type: bool, optional
argument path: run_jdata/training_init_model
Iteration > 0, the model parameters will be initilized from the model trained at the previous iteration. Iteration == 0, the model parameters will be initialized from training_iter0_model_path.
default_training_param:
type: dict
argument path: run_jdata/default_training_param
Training parameters for deepmd-kit in 00.train. You can find instructions from DeePMD-kit documentation.
dp_train_skip_neighbor_stat:
type: bool, optional, default: False
argument path: run_jdata/dp_train_skip_neighbor_stat
Append –skip-neighbor-stat flag to dp train.
dp_compress:
type: bool, optional, default: False
argument path: run_jdata/dp_compress
Use dp compress to compress the model.
training_reuse_iter:
type: int | NoneType, optional
argument path: run_jdata/training_reuse_iter
The minimal index of iteration that continues training models from old models of last iteration.
training_reuse_old_ratio:
type: str | float, optional, default: auto
argument path: run_jdata/training_reuse_old_ratio
The probability proportion of old data during training. It can be:
float: directly assign the probability of old data;
auto:f: automatic probability, where f is the new-to-old ratio;
auto: equivalent to auto:10.
This option is only adopted when continuing training models from old models. This option will override default parameters.
training_reuse_numb_steps:
type: int | NoneType, optional, default: None, alias: training_reuse_stop_batch
argument path: run_jdata/training_reuse_numb_steps
Number of training batch. This option is only adopted when continuing training models from old models. This option will override default parameters.
training_reuse_start_lr:
type: float | NoneType, optional, default: None
argument path: run_jdata/training_reuse_start_lr
The learning rate the start of the training. This option is only adopted when continuing training models from old models. This option will override default parameters.
training_reuse_start_pref_e:
type: int | float | NoneType, optional, default: None
argument path: run_jdata/training_reuse_start_pref_e
The prefactor of energy loss at the start of the training. This option is only adopted when continuing training models from old models. This option will override default parameters.
training_reuse_start_pref_f:
type: int | float | NoneType, optional, default: None
argument path: run_jdata/training_reuse_start_pref_f
The prefactor of force loss at the start of the training. This option is only adopted when continuing training models from old models. This option will override default parameters.
model_devi_activation_func:
type: NoneType | list[list[str]], optional
argument path: run_jdata/model_devi_activation_func
The activation function in the model. The shape of list should be (N_models, 2), where 2 represents the embedding and fitting network. This option will override default parameters.
srtab_file_path:
type: str, optional
argument path: run_jdata/srtab_file_path
The path of the table for the short-range pairwise interaction which is needed when using DP-ZBL potential
one_h5:
type: bool, optional, default: False
argument path: run_jdata/one_h5
When using DeePMD-kit, all of the input data will be merged into one HDF5 file.
training_init_frozen_model:
type: list[str], optional
argument path: run_jdata/training_init_frozen_model
At interation 0, initilize the model parameters from the given frozen models. Number of element should be equal to numb_models.
training_finetune_model:
type: list[str], optional
argument path: run_jdata/training_finetune_model
At interation 0, finetune the model parameters from the given frozen models. Number of element should be equal to numb_models.
fp_task_max:
type: int
argument path: run_jdata/fp_task_max
Maximum number of structures to be calculated in each system in 02.fp of each iteration. If the number of candidate structures exceeds fp_task_max, fp_task_max structures will be randomly picked from the candidates and labeled.
fp_task_min:
type: int
argument path: run_jdata/fp_task_min
Skip the training in the next iteration if the number of structures is no more than fp_task_min.
fp_accurate_threshold:
type: float, optional
argument path: run_jdata/fp_accurate_threshold
If the accurate ratio is larger than this number, no fp calculation will be performed, i.e. fp_task_max = 0.
fp_accurate_soft_threshold:
type: float, optional
argument path: run_jdata/fp_accurate_soft_threshold
If the accurate ratio is between this number and fp_accurate_threshold, the fp_task_max linearly decays to zero.
fp_cluster_vacuum:
type: float, optional
argument path: run_jdata/fp_cluster_vacuum
If the vacuum size is smaller than this value, this cluster will not be chosen for labeling.
detailed_report_make_fp:
type: bool, optional, default: True
argument path: run_jdata/detailed_report_make_fp
If set to true, a detailed report will be generated for each iteration.
ratio_failed:
type: float, optional
argument path: run_jdata/ratio_failed
Check the ratio of unsuccessfully terminated jobs. If too many FP tasks are not converged, RuntimeError will be raised.
Depending on the value of model_devi_engine, different sub args are accepted.
model_devi_engine:
type: str (flag key), default: lammps
argument path: run_jdata/model_devi_engine
possible choices: lammps, amber, calypso, gromacs
Engine for the model deviation task.
When model_devi_engine is set to lammps:
LAMMPS
model_devi_jobs:
type: list
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs
Settings for exploration in 01.model_devi. Each dict in the list corresponds to one iteration. The index of model_devi_jobs exactly accord with index of iterations
This argument takes a list with each element containing the following:
template:
type: dict, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/template
Give an input file template for the supported engine software adopted in 01.model_devi. Through user-defined template, any freedom (function) that is permitted by the engine software could be inherited (invoked) in the workflow.
lmp:
type: str, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/template/lmp
The path to input.lammps template. Instructions can be found in LAMMPS documentation.
plm:
type: str, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/template/plm
The path to input.plumed template. Instructions can be found in PLUMED documentation.
rev_mat:
type: dict, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/rev_mat
revise matrix for revising variable(s) defined in the template into the specific values (iteration-resolved). Values will be broadcasted for all tasks within the iteration invoking this key.
lmp:
type: dict, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/rev_mat/lmp
revise matrix for revising variable(s) defined in the lammps template into the specific values (iteration-resolved).
plm:
type: dict, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/rev_mat/plm
revise matrix for revising variable(s) defined in the plumed template into specific values(iteration-resolved)
sys_rev_mat:
type: dict, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/sys_rev_mat
system-resolved revise matrix for revising variable(s) defined in the template into specific values. Values should be individually assigned to each system adopted by this iteration, through a dictionary where first-level keys are values of sys_idx of this iteration.
sys_idx:
type: list[int]
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/sys_idx
Systems to be selected as the initial structure of MD and be explored. The index corresponds exactly to the sys_configs.
temps:
type: list[float], optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/temps
Temperature (K) in MD.
press:
type: list[float], optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/press
Pressure (Bar) in MD. Required when ensemble is npt.
trj_freq:
type: int
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/trj_freq
Frequecy of trajectory saved in MD.
nsteps:
type: int, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/nsteps
Running steps of MD. It is not optional when not using a template.
nbeads:
type: int, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/nbeads
Number of beads in PIMD. If not given, classical MD will be performed. Only supported for LAMMPS version >= 20230615.
ensemble:
type: str, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/ensemble
Determining which ensemble used in MD, options include “npt” and “nvt”. It is not optional when not using a template.
neidelay:
type: int, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/neidelay
delay building until this many steps since last build.
taut:
type: float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/taut
Coupling time of thermostat (ps).
taup:
type: float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/taup
Coupling time of barostat (ps).
model_devi_f_trust_lo:
type: dict | float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/model_devi_f_trust_lo
Lower bound of forces for the selection. If dict, should be set for each index in sys_idx, respectively.
model_devi_f_trust_hi:
type: dict | float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/model_devi_f_trust_hi
Upper bound of forces for the selection. If dict, should be set for each index in sys_idx, respectively.
model_devi_v_trust_lo:
type: dict | float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/model_devi_v_trust_lo
Lower bound of virial for the selection. If dict, should be set for each index in sys_idx, respectively. Should be used with DeePMD-kit v2.x.
model_devi_v_trust_hi:
type: dict | float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_jobs/model_devi_v_trust_hi
Upper bound of virial for the selection. If dict, should be set for each index in sys_idx, respectively. Should be used with DeePMD-kit v2.x.
model_devi_dt:
type: float
argument path: run_jdata[model_devi_engine=lammps]/model_devi_dt
Timestep for MD. 0.002 is recommend.
model_devi_skip:
type: int
argument path: run_jdata[model_devi_engine=lammps]/model_devi_skip
Number of structures skipped for fp in each MD.
model_devi_f_trust_lo:
type: list[float] | dict | float
argument path: run_jdata[model_devi_engine=lammps]/model_devi_f_trust_lo
Lower bound of forces for the selection. If list or dict, should be set for each index in sys_configs, respectively.
model_devi_f_trust_hi:
type: list[float] | dict | float
argument path: run_jdata[model_devi_engine=lammps]/model_devi_f_trust_hi
Upper bound of forces for the selection. If list or dict, should be set for each index in sys_configs, respectively.
model_devi_v_trust_lo:
type: list[float] | dict | float, optional, default: 10000000000.0
argument path: run_jdata[model_devi_engine=lammps]/model_devi_v_trust_lo
Lower bound of virial for the selection. If list or dict, should be set for each index in sys_configs, respectively. Should be used with DeePMD-kit v2.x.
model_devi_v_trust_hi:
type: list[float] | dict | float, optional, default: 10000000000.0
argument path: run_jdata[model_devi_engine=lammps]/model_devi_v_trust_hi
Upper bound of virial for the selection. If list or dict, should be set for each index in sys_configs, respectively. Should be used with DeePMD-kit v2.x.
model_devi_adapt_trust_lo:
type: bool, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_adapt_trust_lo
Adaptively determines the lower trust levels of force and virial. This option should be used together with model_devi_numb_candi_f, model_devi_numb_candi_v and optionally with model_devi_perc_candi_f and model_devi_perc_candi_v. dpgen will make two sets:
From the frames with force model deviation lower than model_devi_f_trust_hi, select max(model_devi_numb_candi_f, model_devi_perc_candi_f*n_frames) frames with largest force model deviation.
From the frames with virial model deviation lower than model_devi_v_trust_hi, select max(model_devi_numb_candi_v, model_devi_perc_candi_v*n_frames) frames with largest virial model deviation.
The union of the two sets is made as candidate dataset.
model_devi_numb_candi_f:
type: int, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_numb_candi_f
See model_devi_adapt_trust_lo.
model_devi_numb_candi_v:
type: int, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_numb_candi_v
See model_devi_adapt_trust_lo.
model_devi_perc_candi_f:
type: float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_perc_candi_f
See model_devi_adapt_trust_lo.
model_devi_perc_candi_v:
type: float, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_perc_candi_v
See model_devi_adapt_trust_lo.
model_devi_f_avg_relative:
type: bool, optional
argument path: run_jdata[model_devi_engine=lammps]/model_devi_f_avg_relative
Normalized the force model deviations by the RMS force magnitude along the trajectory. This key should not be used with use_relative.
model_devi_clean_traj:
type: bool | int, optional, default: True
argument path: run_jdata[model_devi_engine=lammps]/model_devi_clean_traj
If type of model_devi_clean_traj is bool type then it denote whether to clean traj folders in MD since they are too large. If it is Int type, then the most recent n iterations of traj folders will be retained, others will be removed.
model_devi_merge_traj:
type: bool, optional, default: False
argument path: run_jdata[model_devi_engine=lammps]/model_devi_merge_traj
If model_devi_merge_traj is set as True, only all.lammpstrj will be generated, instead of lots of small traj files.
model_devi_nopbc:
type: bool, optional, default: False
argument path: run_jdata[model_devi_engine=lammps]/model_devi_nopbc
Assume open boundary condition in MD simulations.
model_devi_plumed:
type: bool, optional, default: False
argument path: run_jdata[model_devi_engine=lammps]/model_devi_plumed
model_devi_plumed_path:
type: bool, optional, default: False
argument path: run_jdata[model_devi_engine=lammps]/model_devi_plumed_path
shuffle_poscar:
type: bool, optional, default: False
argument path: run_jdata[model_devi_engine=lammps]/shuffle_poscar
Shuffle atoms of each frame before running simulations. The purpose is to sample the element occupation of alloys.
use_relative:
type: bool, optional, default: False
argument path: run_jdata[model_devi_engine=lammps]/use_relative
Calculate relative force model deviation.
epsilon:
type: float, optional
argument path: run_jdata[model_devi_engine=lammps]/epsilon
The level parameter for computing the relative force model deviation.
use_relative_v:
type: bool, optional, default: False
argument path: run_jdata[model_devi_engine=lammps]/use_relative_v
Calculate relative virial model deviation.
epsilon_v:
type: float, optional
argument path: run_jdata[model_devi_engine=lammps]/epsilon_v
The level parameter for computing the relative virial model deviation.
When model_devi_engine is set to amber:
Amber DPRc engine. The command argument in the machine file should be path to sander.
model_devi_jobs:
type: list
argument path: run_jdata[model_devi_engine=amber]/model_devi_jobs
List of dicts. The list including the dict for information of each cycle.
This argument takes a list with each element containing the following:
sys_idx:
type: list[int]
argument path: run_jdata[model_devi_engine=amber]/model_devi_jobs/sys_idx
List of ints. List of systems to run.
trj_freq:
type: int
argument path: run_jdata[model_devi_engine=amber]/model_devi_jobs/trj_freq
Frequency to dump trajectory.
restart_from_iter:
type: int, optional
argument path: run_jdata[model_devi_engine=amber]/model_devi_jobs/restart_from_iter
The iteration index to restart the simulation from. If not given, the simulation is restarted from sys_configs.
low_level:
type: str
argument path: run_jdata[model_devi_engine=amber]/low_level
Low level method. The value will be filled into mdin file as @qm_theory@.
cutoff:
type: float
argument path: run_jdata[model_devi_engine=amber]/cutoff
Cutoff radius for the DPRc model.
parm7_prefix:
type: str, optional
argument path: run_jdata[model_devi_engine=amber]/parm7_prefix
The path prefix to AMBER PARM7 files.
parm7:
type: list[str]
argument path: run_jdata[model_devi_engine=amber]/parm7
List of paths to AMBER PARM7 files. Each file maps to a system.
mdin_prefix:
type: str, optional
argument path: run_jdata[model_devi_engine=amber]/mdin_prefix
The path prefix to AMBER mdin template files.
mdin:
type: list[str]
argument path: run_jdata[model_devi_engine=amber]/mdin
List of paths to AMBER mdin template files. Each files maps to a system. In the template, the following keywords will be replaced by the actual value: @freq@: freq to dump trajectory; @nstlim@: total time step to run; @qm_region@: AMBER mask of the QM region; @qm_theory@: The low level QM theory, such as DFTB2; @qm_charge@: The total charge of the QM theory, such as -2; @rcut@: cutoff radius of the DPRc model; @GRAPH_FILE0@, @GRAPH_FILE1@, … : graph files.
qm_region:
type: list[str]
argument path: run_jdata[model_devi_engine=amber]/qm_region
List of strings. AMBER mask of the QM region. Each mask maps to a system.
qm_charge:
type: list[int]
argument path: run_jdata[model_devi_engine=amber]/qm_charge
List of ints. Charge of the QM region. Each charge maps to a system.
nsteps:
type: list[int]
argument path: run_jdata[model_devi_engine=amber]/nsteps
List of ints. The number of steps to run. Each number maps to a system.
r:
type: list[list[typing.Union[float, list[float]]]]
argument path: run_jdata[model_devi_engine=amber]/r
2D or 3D list of floats. Constrict values for the enhanced sampling. The first dimension maps to systems. The second dimension maps to confs in each system. The third dimension is the constrict value. It can be a single float for 1D or list of floats for nD.
disang_prefix:
type: str, optional
argument path: run_jdata[model_devi_engine=amber]/disang_prefix
The path prefix to disang prefix.
disang:
type: list[str]
argument path: run_jdata[model_devi_engine=amber]/disang
List of paths to AMBER disang files. Each file maps to a sytem. The keyword RVAL will be replaced by the constrict values, or RVAL1, RVAL2, … for an nD system.
model_devi_f_trust_lo:
type: list[float] | dict | float
argument path: run_jdata[model_devi_engine=amber]/model_devi_f_trust_lo
Lower bound of forces for the selection. If dict, should be set for each index in sys_idx, respectively.
model_devi_f_trust_hi:
type: list[float] | dict | float
argument path: run_jdata[model_devi_engine=amber]/model_devi_f_trust_hi
Upper bound of forces for the selection. If dict, should be set for each index in sys_idx, respectively.
When model_devi_engine is set to calypso:
TODO: add doc
When model_devi_engine is set to gromacs:
TODO: add doc
Depending on the value of fp_style, different sub args are accepted.
fp_style:
type: str (flag key)
argument path: run_jdata/fp_style
possible choices: vasp, gaussian, siesta, cp2k, abacus, amber/diff, pwmat, pwscf, custom
Software for First Principles.
When fp_style is set to vasp:
fp_pp_path:
type: str
argument path: run_jdata[fp_style=vasp]/fp_pp_path
Directory of psuedo-potential file to be used for 02.fp exists.
fp_pp_files:
type: list[str]
argument path: run_jdata[fp_style=vasp]/fp_pp_files
Psuedo-potential file to be used for 02.fp. Note that the order of elements should correspond to the order in type_map.
fp_incar:
type: str
argument path: run_jdata[fp_style=vasp]/fp_incar
Input file for VASP. INCAR must specify KSPACING and KGAMMA.
fp_aniso_kspacing:
type: list[float], optional
argument path: run_jdata[fp_style=vasp]/fp_aniso_kspacing
Set anisotropic kspacing. Usually useful for 1-D or 2-D materials. Only support VASP. If it is setting the KSPACING key in INCAR will be ignored.
cvasp:
type: bool, optional
argument path: run_jdata[fp_style=vasp]/cvasp
If cvasp is true, DP-GEN will use Custodian to help control VASP calculation.
fp_skip_bad_box:
type: str, optional
argument path: run_jdata[fp_style=vasp]/fp_skip_bad_box
Skip the configurations that are obviously unreasonable before 02.fp
When fp_style is set to gaussian:
use_clusters:
type: bool, optional, default: False
argument path: run_jdata[fp_style=gaussian]/use_clusters
If set to true, clusters will be taken instead of the whole system.
cluster_cutoff:
type: float, optional
argument path: run_jdata[fp_style=gaussian]/cluster_cutoff
The soft cutoff radius of clusters if use_clusters is set to true. Molecules will be taken as whole even if part of atoms is out of the cluster. Use cluster_cutoff_hard to only take atoms within the hard cutoff radius.
cluster_cutoff_hard:
type: float, optional
argument path: run_jdata[fp_style=gaussian]/cluster_cutoff_hard
The hard cutoff radius of clusters if use_clusters is set to true. Outside the hard cutoff radius, atoms will not be taken even if they are in a molecule where some atoms are within the cutoff radius.
cluster_minify:
type: bool, optional, default: False
argument path: run_jdata[fp_style=gaussian]/cluster_minify
If enabled, when an atom within the soft cutoff radius connects a single bond with a non-hydrogen atom out of the soft cutoff radius, the outer atom will be replaced by a hydrogen atom. When the outer atom is a hydrogen atom, the outer atom will be kept. In this case, other atoms out of the soft cutoff radius will be removed.
fp_params:
type: dict
argument path: run_jdata[fp_style=gaussian]/fp_params
Parameters for Gaussian calculation.
keywords:
type: list[str] | str
argument path: run_jdata[fp_style=gaussian]/fp_params/keywords
Keywords for Gaussian input, e.g. force b3lyp/6-31g**. If a list, run multiple steps.
multiplicity:
type: str | int, optional, default: auto
argument path: run_jdata[fp_style=gaussian]/fp_params/multiplicity
Spin multiplicity for Gaussian input. If auto, multiplicity will be detected automatically, with the following rules: when fragment_guesses=True, multiplicity will +1 for each radical, and +2 for each oxygen molecule; when fragment_guesses=False, multiplicity will be 1 or 2, but +2 for each oxygen molecule.
nproc:
type: int
argument path: run_jdata[fp_style=gaussian]/fp_params/nproc
The number of processors for Gaussian input.
charge:
type: int, optional, default: 0
argument path: run_jdata[fp_style=gaussian]/fp_params/charge
Molecule charge. Only used when charge is not provided by the system.
fragment_guesses:
type: bool, optional, default: False
argument path: run_jdata[fp_style=gaussian]/fp_params/fragment_guesses
Initial guess generated from fragment guesses. If True, multiplicity should be auto.
basis_set:
type: str, optional
argument path: run_jdata[fp_style=gaussian]/fp_params/basis_set
Custom basis set.
keywords_high_multiplicity:
type: str, optional
argument path: run_jdata[fp_style=gaussian]/fp_params/keywords_high_multiplicity
Keywords for points with multiple raicals. multiplicity should be auto. If not set, fallback to normal keywords.
When fp_style is set to siesta:
use_clusters:
type: bool, optional
argument path: run_jdata[fp_style=siesta]/use_clusters
If set to true, clusters will be taken instead of the whole system. This option does not work with DeePMD-kit 0.x.
cluster_cutoff:
type: float, optional
argument path: run_jdata[fp_style=siesta]/cluster_cutoff
The cutoff radius of clusters if use_clusters is set to true.
fp_params:
type: dict
argument path: run_jdata[fp_style=siesta]/fp_params
Parameters for siesta calculation.
ecut:
type: int
argument path: run_jdata[fp_style=siesta]/fp_params/ecut
Define the plane wave cutoff for grid.
ediff:
type: float
argument path: run_jdata[fp_style=siesta]/fp_params/ediff
Tolerance of Density Matrix.
kspacing:
type: float
argument path: run_jdata[fp_style=siesta]/fp_params/kspacing
Sample factor in Brillouin zones.
mixingWeight:
type: float
argument path: run_jdata[fp_style=siesta]/fp_params/mixingWeight
Proportion a of output Density Matrix to be used for the input Density Matrix of next SCF cycle (linear mixing).
NumberPulay:
type: int
argument path: run_jdata[fp_style=siesta]/fp_params/NumberPulay
Controls the Pulay convergence accelerator.
fp_pp_path:
type: str
argument path: run_jdata[fp_style=siesta]/fp_pp_path
Directory of psuedo-potential or numerical orbital files to be used for 02.fp exists.
fp_pp_files:
type: list[str]
argument path: run_jdata[fp_style=siesta]/fp_pp_files
Psuedo-potential file to be used for 02.fp. Note that the order of elements should correspond to the order in type_map.
When fp_style is set to cp2k:
user_fp_params:
type: dict, optional, alias: fp_params
argument path: run_jdata[fp_style=cp2k]/user_fp_params
Parameters for cp2k calculation. find detail in manual.cp2k.org. only the kind section must be set before use. we assume that you have basic knowledge for cp2k input.
external_input_path:
type: str, optional
argument path: run_jdata[fp_style=cp2k]/external_input_path
Conflict with key:user_fp_params. enable the template input provided by user. some rules should be followed, read the following text in detail:
One must present a KEYWORD ABC in the section CELL so that the DP-GEN can replace the cell on-the-fly.
One need to add these lines under FORCE_EVAL section to print forces and stresses:
STRESS_TENSOR ANALYTICAL
  &PRINT
    &FORCES ON
    &END FORCES
    &STRESS_TENSOR ON
    &END STRESS_TENSOR
  &END PRINT
When fp_style is set to abacus:
fp_pp_path:
type: str
argument path: run_jdata[fp_style=abacus]/fp_pp_path
Directory of psuedo-potential or numerical orbital files to be used for 02.fp exists.
fp_pp_files:
type: list[str]
argument path: run_jdata[fp_style=abacus]/fp_pp_files
Psuedo-potential file to be used for 02.fp. Note that the order of elements should correspond to the order in type_map.
fp_orb_files:
type: list[str], optional
argument path: run_jdata[fp_style=abacus]/fp_orb_files
numerical orbital file to be used for 02.fp when using LCAO basis. Note that the order of elements should correspond to the order in type_map.
fp_incar:
type: str, optional
argument path: run_jdata[fp_style=abacus]/fp_incar
Input file for ABACUS. This is optinal but the priority is lower than user_fp_params, and you should not set user_fp_params if you want to use fp_incar.
fp_kpt_file:
type: str, optional
argument path: run_jdata[fp_style=abacus]/fp_kpt_file
KPT file for ABACUS.If the “kspacing” or “gamma_only=1” is defined in INPUT or “k_points” is defined, fp_kpt_file will be ignored.
fp_dpks_descriptor:
type: str, optional
argument path: run_jdata[fp_style=abacus]/fp_dpks_descriptor
DeePKS descriptor file name. The file should be in pseudopotential directory.
user_fp_params:
type: dict, optional
argument path: run_jdata[fp_style=abacus]/user_fp_params
Set the key and value of INPUT.
k_points:
type: list[int], optional
argument path: run_jdata[fp_style=abacus]/k_points
Monkhorst-Pack k-grids setting for generating KPT file of ABACUS, such as: [1,1,1,0,0,0]. NB: if “kspacing” or “gamma_only=1” is defined in INPUT, k_points will be ignored.
When fp_style is set to amber/diff:
Amber/diff style for DPRc models. Note: this fp style only supports to be used with model_devi_engine amber, where some arguments are reused. The command argument in the machine file should be path to sander. One should also install dpamber and make it visible in the PATH.
high_level:
type: str
argument path: run_jdata[fp_style=amber/diff]/high_level
High level method. The value will be filled into mdin template as @qm_theory@.
fp_params:
type: dict
argument path: run_jdata[fp_style=amber/diff]/fp_params
Parameters for FP calculation.
high_level_mdin:
type: str
argument path: run_jdata[fp_style=amber/diff]/fp_params/high_level_mdin
Path to high-level AMBER mdin template file. %qm_theory%, %qm_region%, and %qm_charge% will be replaced.
low_level_mdin:
type: str
argument path: run_jdata[fp_style=amber/diff]/fp_params/low_level_mdin
Path to low-level AMBER mdin template file. %qm_theory%, %qm_region%, and %qm_charge% will be replaced.
When fp_style is set to pwmat:
TODO: add doc
When fp_style is set to pwscf:
fp_pp_path:
type: str
argument path: run_jdata[fp_style=pwscf]/fp_pp_path
Directory of psuedo-potential file to be used for 02.fp exists.
fp_pp_files:
type: list[str]
argument path: run_jdata[fp_style=pwscf]/fp_pp_files
Psuedo-potential file to be used for 02.fp. Note that the order of elements should correspond to the order in type_map.
fp_params:
type: dict, optional
argument path: run_jdata[fp_style=pwscf]/fp_params
Parameters for pwscf calculation. It has lower priority than user_fp_params.
ecut:
type: float
argument path: run_jdata[fp_style=pwscf]/fp_params/ecut
ecutwfc in pwscf.
ediff:
type: float
argument path: run_jdata[fp_style=pwscf]/fp_params/ediff
conv_thr and ts_vdw_econv_thr in pwscf.
smearing:
type: str
argument path: run_jdata[fp_style=pwscf]/fp_params/smearing
smearing in pwscf.
sigma:
type: float
argument path: run_jdata[fp_style=pwscf]/fp_params/sigma
degauss in pwscf.
kspacing:
type: float
argument path: run_jdata[fp_style=pwscf]/fp_params/kspacing
The spacing between kpoints. Helps to determin KPOINTS in pwscf.
user_fp_params:
type: dict, optional
argument path: run_jdata[fp_style=pwscf]/user_fp_params
Parameters for pwscf calculation. Find details at https://www.quantum-espresso.org/Doc/INPUT_PW.html. When user_fp_params is set, the settings in fp_params will be ignored. If one wants to use user_fp_params, kspacing must be set in user_fp_params. kspacing is the spacing between kpoints, and helps to determin KPOINTS in pwscf.
When fp_style is set to custom:
Custom FP code. You need to provide the input and output file format and name. The command argument in the machine file should be the script to run custom FP codes. The extra forward and backward files can be defined in the machine file.
fp_params:
type: dict
argument path: run_jdata[fp_style=custom]/fp_params
Parameters for FP calculation.
input_fmt:
type: str
argument path: run_jdata[fp_style=custom]/fp_params/input_fmt
Input dpdata format of the custom FP code. Such format should only need the first argument as the file name.
input_fn:
type: str
argument path: run_jdata[fp_style=custom]/fp_params/input_fn
Input file name of the custom FP code.
output_fmt:
type: str
argument path: run_jdata[fp_style=custom]/fp_params/output_fmt
Output dpata format of the custom FP code. Such format should only need the first argument as the file name.
output_fn:
type: str
argument path: run_jdata[fp_style=custom]/fp_params/output_fn
Output file name of the custom FP code.

dpgen run machine parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

run_mdata:

type: dict
argument path: run_mdata
machine.json file
api_version:
type: str, optional, default: 1.0
argument path: run_mdata/api_version
Please set to 1.0
deepmd_version:
type: str, optional, default: 2
argument path: run_mdata/deepmd_version
DeePMD-kit version, e.g. 2.1.3
train:
type: dict
argument path: run_mdata/train
Parameters of command, machine, and resources for train
command:
type: str
argument path: run_mdata/train/command
Command of a program.
machine:
type: dict
argument path: run_mdata/train/machine
batch_type:
type: str
argument path: run_mdata/train/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: run_mdata/train/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: run_mdata/train/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: run_mdata/train/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: run_mdata/train/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: run_mdata/train/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: run_mdata/train/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: run_mdata/train/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: run_mdata/train/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: run_mdata/train/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: run_mdata/train/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: run_mdata/train/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: run_mdata/train/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: run_mdata/train/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: run_mdata/train/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: run_mdata/train/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: run_mdata/train/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: run_mdata/train/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: run_mdata/train/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: run_mdata/train/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: run_mdata/train/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: run_mdata/train/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: run_mdata/train/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: run_mdata/train/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: run_mdata/train/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: run_mdata/train/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: run_mdata/train/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
input_data:
type: dict
argument path: run_mdata/train/machine[BohriumContext]/remote_profile/input_data
Configuration of job
resources:
type: dict
argument path: run_mdata/train/resources
number_node:
type: int, optional, default: 1
argument path: run_mdata/train/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: run_mdata/train/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: run_mdata/train/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: run_mdata/train/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: run_mdata/train/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: run_mdata/train/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: run_mdata/train/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: run_mdata/train/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: run_mdata/train/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: run_mdata/train/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: run_mdata/train/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/train/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: run_mdata/train/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/train/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/train/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: run_mdata/train/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: run_mdata/train/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: run_mdata/train/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: run_mdata/train/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: run_mdata/train/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/train/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: run_mdata/train/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: run_mdata/train/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: run_mdata/train/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: run_mdata/train/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/train/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/train/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: run_mdata/train/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: run_mdata/train/resources[Shell]/kwargs
This field is empty for this batch.
user_forward_files:
type: list, optional
argument path: run_mdata/train/user_forward_files
Files to be forwarded to the remote machine.
user_backward_files:
type: list, optional
argument path: run_mdata/train/user_backward_files
Files to be backwarded from the remote machine.
model_devi:
type: dict
argument path: run_mdata/model_devi
Parameters of command, machine, and resources for model_devi
command:
type: str
argument path: run_mdata/model_devi/command
Command of a program.
machine:
type: dict
argument path: run_mdata/model_devi/machine
batch_type:
type: str
argument path: run_mdata/model_devi/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: run_mdata/model_devi/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: run_mdata/model_devi/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: run_mdata/model_devi/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: run_mdata/model_devi/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: run_mdata/model_devi/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: run_mdata/model_devi/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: run_mdata/model_devi/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: run_mdata/model_devi/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: run_mdata/model_devi/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
input_data:
type: dict
argument path: run_mdata/model_devi/machine[BohriumContext]/remote_profile/input_data
Configuration of job
resources:
type: dict
argument path: run_mdata/model_devi/resources
number_node:
type: int, optional, default: 1
argument path: run_mdata/model_devi/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: run_mdata/model_devi/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: run_mdata/model_devi/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: run_mdata/model_devi/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: run_mdata/model_devi/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: run_mdata/model_devi/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: run_mdata/model_devi/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: run_mdata/model_devi/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: run_mdata/model_devi/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: run_mdata/model_devi/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: run_mdata/model_devi/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/model_devi/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: run_mdata/model_devi/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/model_devi/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/model_devi/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: run_mdata/model_devi/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: run_mdata/model_devi/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: run_mdata/model_devi/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: run_mdata/model_devi/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: run_mdata/model_devi/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/model_devi/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: run_mdata/model_devi/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: run_mdata/model_devi/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: run_mdata/model_devi/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: run_mdata/model_devi/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/model_devi/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/model_devi/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: run_mdata/model_devi/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: run_mdata/model_devi/resources[Shell]/kwargs
This field is empty for this batch.
user_forward_files:
type: list, optional
argument path: run_mdata/model_devi/user_forward_files
Files to be forwarded to the remote machine.
user_backward_files:
type: list, optional
argument path: run_mdata/model_devi/user_backward_files
Files to be backwarded from the remote machine.
fp:
type: dict
argument path: run_mdata/fp
Parameters of command, machine, and resources for fp
command:
type: str
argument path: run_mdata/fp/command
Command of a program.
machine:
type: dict
argument path: run_mdata/fp/machine
batch_type:
type: str
argument path: run_mdata/fp/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: run_mdata/fp/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: run_mdata/fp/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: run_mdata/fp/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: run_mdata/fp/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: run_mdata/fp/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: run_mdata/fp/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: run_mdata/fp/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: run_mdata/fp/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: run_mdata/fp/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: run_mdata/fp/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
input_data:
type: dict
argument path: run_mdata/fp/machine[BohriumContext]/remote_profile/input_data
Configuration of job
resources:
type: dict
argument path: run_mdata/fp/resources
number_node:
type: int, optional, default: 1
argument path: run_mdata/fp/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: run_mdata/fp/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: run_mdata/fp/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: run_mdata/fp/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: run_mdata/fp/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: run_mdata/fp/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: run_mdata/fp/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: run_mdata/fp/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: run_mdata/fp/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: run_mdata/fp/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: run_mdata/fp/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/fp/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: run_mdata/fp/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/fp/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: run_mdata/fp/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: run_mdata/fp/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: run_mdata/fp/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: run_mdata/fp/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: run_mdata/fp/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: run_mdata/fp/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/fp/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: run_mdata/fp/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: run_mdata/fp/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: run_mdata/fp/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: run_mdata/fp/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/fp/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: run_mdata/fp/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: run_mdata/fp/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: run_mdata/fp/resources[Shell]/kwargs
This field is empty for this batch.
user_forward_files:
type: list, optional
argument path: run_mdata/fp/user_forward_files
Files to be forwarded to the remote machine.
user_backward_files:
type: list, optional
argument path: run_mdata/fp/user_backward_files
Files to be backwarded from the remote machine.

Init

Init_bulk

You may prepare initial data for bulk systems with VASP by:

dpgen init_bulk PARAM [MACHINE]

The MACHINE configure file is optional. If this parameter exists, then the optimization tasks or MD tasks will be submitted automatically according to MACHINE.json.

Basically init_bulk can be divided into four parts , denoted as stages in PARAM:

Relax in folder 00.place_ele
Perturb and scale in folder 01.scale_pert
Run a short AIMD in folder 02.md
Collect data in folder 02.md.

All stages must be in order. One doesn’t need to run all stages. For example, you may run stage 1 and 2, generating supercells as starting point of exploration in dpgen run.

If MACHINE is None, there should be only one stage in stages. Corresponding tasks will be generated, but user’s intervention should be involved in, to manually run the scripts.

Following is an example for PARAM, which generates data from a typical structure hcp.

{
    "stages" : [1,2,3,4],
    "cell_type":    "hcp",
    "latt":     4.479,
    "super_cell":   [2, 2, 2],
    "elements":     ["Mg"],
    "potcars":      ["....../POTCAR"],
    "relax_incar": "....../INCAR_metal_rlx",
    "md_incar" : "....../INCAR_metal_md",
    "scale":        [1.00],
    "skip_relax":   false,
    "pert_numb":    2,
    "md_nstep" : 5,
    "pert_box":     0.03,
    "pert_atom":    0.01,
    "coll_ndata":   5000,
    "type_map" : [ "Mg", "Al"],
    "_comment":     "that's all"
}

If you want to specify a structure as starting point for init_bulk, you may set in PARAM as follows.

"from_poscar":	true,
"from_poscar_path":	"....../C_mp-47_conventional.POSCAR",

init_bulk supports both VASP and ABACUS for first-principle calculation. You can choose the software by specifying the key init_fp_style. If init_fp_style is not specified, the default software will be VASP.

When using ABACUS for init_fp_style, the keys of the paths of INPUT files for relaxation and MD simulations are the same as INCAR for VASP, which are relax_incar and md_incar respectively. Use relax_kpt and md_kpt for the relative path for KPT files of relaxation and MD simulations. They two can be omitted if kspacing (in unit of 1/Bohr) or gamma_only has been set in corresponding INPUT files. If from_poscar is set to false, you have to specify atom_masses in the same order as elements.

dpgen init_bulk parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

init_bulk_jdata:

type: dict
argument path: init_bulk_jdata
Generate initial data for bulk systems.
stages:
type: list[int]
argument path: init_bulk_jdata/stages
Stages for init_bulk.
elements:
type: list[str]
argument path: init_bulk_jdata/elements
Atom types.
potcars:
type: list[str], optional
argument path: init_bulk_jdata/potcars
Path of POTCAR.
cell_type:
type: str, optional
argument path: init_bulk_jdata/cell_type
Specifying which typical structure to be generated. Options include fcc, hcp, bcc, sc, diamond.
super_cell:
type: list[int]
argument path: init_bulk_jdata/super_cell
Size of supercell.
from_poscar:
type: bool, optional, default: False
argument path: init_bulk_jdata/from_poscar
Deciding whether to use a given poscar as the beginning of relaxation. If it’s true, keys (cell_type, latt) will be aborted. Otherwise, these two keys are necessary.
from_poscar_path:
type: str, optional
argument path: init_bulk_jdata/from_poscar_path
Path of POSCAR for VASP or STRU for ABACUS. Necessary if from_poscar is true.
relax_incar:
type: str, optional
argument path: init_bulk_jdata/relax_incar
Path of INCAR for VASP or INPUT for ABACUS for relaxation in VASP. Necessary if stages include 1.
md_incar:
type: str, optional
argument path: init_bulk_jdata/md_incar
Path of INCAR for VASP or INPUT for ABACUS for MD in VASP. Necessary if stages include 3.
scale:
type: list[float]
argument path: init_bulk_jdata/scale
Scales for isotropic transforming cells.
skip_relax:
type: bool
argument path: init_bulk_jdata/skip_relax
If it’s true, you may directly run stage 2 (perturb and scale) using an unrelaxed POSCAR.
pert_numb:
type: int
argument path: init_bulk_jdata/pert_numb
Number of perturbations for each scaled (key scale) POSCAR.
pert_box:
type: float
argument path: init_bulk_jdata/pert_box
Anisotropic Perturbation for cells (independent changes of lengths of three box vectors as well as angel among) in decimal formats. 9 elements of the 3x3 perturbation matrix will be randomly sampled from a uniform distribution (default) in the range [-pert_box, pert_box]. Such a perturbation matrix adds the identity matrix gives the actual transformation matrix for this perturbation operation.
pert_atom:
type: float
argument path: init_bulk_jdata/pert_atom
Perturbation of atom coordinates (Angstrom). Random perturbations are performed on three coordinates of each atom by adding values randomly sampled from a uniform distribution in the range [-pert_atom, pert_atom].
md_nstep:
type: int
argument path: init_bulk_jdata/md_nstep
Steps of AIMD in stage 3. If it’s not equal to settings via NSW in md_incar, DP-GEN will follow NSW.
coll_ndata:
type: int
argument path: init_bulk_jdata/coll_ndata
Maximal number of collected data.
type_map:
type: list[str], optional
argument path: init_bulk_jdata/type_map
The indices of elements in deepmd formats will be set in this order.
Depending on the value of init_fp_style, different sub args are accepted.
init_fp_style:
type: str (flag key), default: VASP
argument path: init_bulk_jdata/init_fp_style
possible choices: VASP, ABACUS
First-principle software. If this key is absent.
When init_fp_style is set to VASP:
No more parameters is needed to be added.
When init_fp_style is set to ABACUS:
ABACUS
relax_kpt:
type: str, optional
argument path: init_bulk_jdata[ABACUS]/relax_kpt
Path of KPT file for relaxation in stage 1. Only useful if init_fp_style is “ABACUS”.
md_kpt:
type: str, optional
argument path: init_bulk_jdata[ABACUS]/md_kpt
Path of KPT file for MD simulations in stage 3. Only useful if init_fp_style is “ABACUS”.
atom_masses:
type: list[float], optional
argument path: init_bulk_jdata[ABACUS]/atom_masses
List of atomic masses of elements. The order should be the same as Elements. Only useful if init_fp_style is “ABACUS”.

dpgen init_bulk machine parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

init_bulk_mdata:

type: dict
argument path: init_bulk_mdata
machine.json file
api_version:
type: str, optional, default: 1.0
argument path: init_bulk_mdata/api_version
Please set to 1.0
deepmd_version:
type: str, optional, default: 2
argument path: init_bulk_mdata/deepmd_version
DeePMD-kit version, e.g. 2.1.3
fp:
type: dict
argument path: init_bulk_mdata/fp
Parameters of command, machine, and resources for fp
command:
type: str
argument path: init_bulk_mdata/fp/command
Command of a program.
machine:
type: dict
argument path: init_bulk_mdata/fp/machine
batch_type:
type: str
argument path: init_bulk_mdata/fp/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: init_bulk_mdata/fp/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: init_bulk_mdata/fp/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: init_bulk_mdata/fp/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: init_bulk_mdata/fp/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: init_bulk_mdata/fp/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: init_bulk_mdata/fp/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: init_bulk_mdata/fp/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: init_bulk_mdata/fp/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: init_bulk_mdata/fp/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
input_data:
type: dict
argument path: init_bulk_mdata/fp/machine[BohriumContext]/remote_profile/input_data
Configuration of job
resources:
type: dict
argument path: init_bulk_mdata/fp/resources
number_node:
type: int, optional, default: 1
argument path: init_bulk_mdata/fp/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: init_bulk_mdata/fp/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: init_bulk_mdata/fp/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: init_bulk_mdata/fp/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: init_bulk_mdata/fp/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: init_bulk_mdata/fp/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: init_bulk_mdata/fp/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: init_bulk_mdata/fp/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: init_bulk_mdata/fp/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: init_bulk_mdata/fp/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: init_bulk_mdata/fp/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: init_bulk_mdata/fp/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: init_bulk_mdata/fp/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: init_bulk_mdata/fp/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: init_bulk_mdata/fp/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: init_bulk_mdata/fp/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: init_bulk_mdata/fp/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: init_bulk_mdata/fp/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: init_bulk_mdata/fp/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: init_bulk_mdata/fp/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_bulk_mdata/fp/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: init_bulk_mdata/fp/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: init_bulk_mdata/fp/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: init_bulk_mdata/fp/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: init_bulk_mdata/fp/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_bulk_mdata/fp/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_bulk_mdata/fp/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: init_bulk_mdata/fp/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: init_bulk_mdata/fp/resources[Shell]/kwargs
This field is empty for this batch.
user_forward_files:
type: list, optional
argument path: init_bulk_mdata/fp/user_forward_files
Files to be forwarded to the remote machine.
user_backward_files:
type: list, optional
argument path: init_bulk_mdata/fp/user_backward_files
Files to be backwarded from the remote machine.

Init_surf

You may prepare initial data for surface systems with VASP by:

dpgen init_surf PARAM [MACHINE]

The MACHINE configure file is optional. If this parameter exists, then the optimization tasks or MD tasks will be submitted automatically according to MACHINE.json. That is to say, if one only wants to prepare surf-xxx/sys-xxx folders for the second stage but wants to skip relaxation, dpgen init_surf PARAM should be used (without MACHINE). “stages” and “skip_relax” in PARAM should be set as:

  "stages": [1,2],
  "skip_relax": true,

Basically init_surf can be divided into two parts , denoted as stages in PARAM:

Build specific surface in folder 00.place_ele
Pertub and scale in folder 01.scale_pert

All stages must be in order.

Generally, init_surf does not run AIMD but only generates a lot of configurations. Compared with init_bulk, which runs DFT calculations twice, init_surf does once. Usually, we do init_bulk, run many rounds of DP-GEN iterations, collect enough data for the bulk system, and do init_surf after that. At this point, the lattice constant has been determined, and the lattice constant required for the initial configuration of init_surf can be used directly. These configurations made by init_surf are prepared for 01.model_devi. Candidates will do DFT calculation in 02.fp.

Generate vacuum layers

According to the source code of pert_scaled, init_surf will generate a series of surface structures with specified separations between the sample layer and its periodic image. There are two ways to specify the interval in generating the vacuum layers: 1) to set the interval value and 2) to set the number of intervals.

You can use layer_numb (the number of layers of the slab) or z_min (the total thickness) to specify the thickness of the atoms below. Then vacuum_* parameters specify the vacuum layers above. dpgen init_surf will make a series of structures with the thickness of vacuum layers from vacuum_min to vacuum_max. The number of vacuum layers is controlled by the parameter vacuum_resol.

The layers will be generated even when the size of vacuum_resol is 1. When the size of vacuum_resol is 2 or it is empty, the whole interval range is divided into the nearby region with denser intervals (head region) and the far-away region with sparser intervals (tail region), which are divided by mid_point.

When the size of vacuum_resol is 2, two elements respectively decide the number of intervals in head region and tail region.

When vacuum_resol is empty, the number of intervals in the head region = vacuum_num * head_ratio. vacuum_num and head_ratio are both keys in param.json.

Attach files in the task path

One can use the machine parameter forward_files to upload other files besides POSCAR, INCAR, and POTCAR. For example, “vdw_kernal.bindat” for each task.

See the document of task parameters.

Following is an example for PARAM, which generates data from a typical structure fcc.

{
  "stages": [
    1,
    2
  ],
  "cell_type": "fcc",
  "latt": 4.034,
  "super_cell": [
    2,
    2,
    2
  ],
  "layer_numb": 3,
  "vacuum_max": 9.0,
  "vacuum_resol": [
    0.5,
    1
  ],
  "mid_point": 4.0,
  "millers": [
    [
      1,
      0,
      0
    ],
    [
      1,
      1,
      0
    ],
    [
      1,
      1,
      1
    ]
  ],
  "elements": [
    "Al"
  ],
  "potcars": [
    "....../POTCAR"
  ],
  "relax_incar": "....../INCAR_metal_rlx_low",
  "scale": [
    1.0
  ],
  "skip_relax": true,
  "pert_numb": 2,
  "pert_box": 0.03,
  "pert_atom": 0.01,
  "_comment": "that's all"
}

Another example is from_poscar method. Here you need to specify the POSCAR file.

{
  "stages": [
    1,
    2
  ],
  "cell_type": "fcc",
  "from_poscar":	true,
  "from_poscar_path":	"POSCAR",
  "super_cell": [
    1,
    1,
    1
  ],
  "layer_numb": 3,
  "vacuum_max": 5.0,
  "vacuum_resol": [0.5,2],
  "mid_point": 2.0,
  "millers": [
    [
      1,
      0,
      0
    ]
  ],
  "elements": [
    "Al"
  ],
  "potcars": [
    "./POTCAR"
  ],
  "relax_incar" : "INCAR_metal_rlx_low",
  "scale": [
    1.0
  ],
  "skip_relax": true,
  "pert_numb": 5,
  "pert_box": 0.03,
  "pert_atom": 0.01,
  "coll_ndata": 5000,
  "_comment": "that's all"
}

dpgen init_surf parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

init_surf_jdata:

type: dict
argument path: init_surf_jdata
Generate initial data for surface systems.
stages:
type: list[int]
argument path: init_surf_jdata/stages
Stages for init_surf.
elements:
type: list[str]
argument path: init_surf_jdata/elements
Atom types.
potcars:
type: list[str], optional
argument path: init_surf_jdata/potcars
Path of POTCAR.
cell_type:
type: str, optional
argument path: init_surf_jdata/cell_type
Specifying which typical structure to be generated. Options include fcc, hcp, bcc, sc, diamond.
super_cell:
type: list[int]
argument path: init_surf_jdata/super_cell
Size of supercell.
from_poscar:
type: bool, optional, default: False
argument path: init_surf_jdata/from_poscar
Deciding whether to use a given poscar as the beginning of relaxation. If it’s true, keys (cell_type, latt) will be aborted. Otherwise, these two keys are necessary.
from_poscar_path:
type: str, optional
argument path: init_surf_jdata/from_poscar_path
Path of POSCAR for VASP or STRU for ABACUS. Necessary if from_poscar is true.
latt:
type: float
argument path: init_surf_jdata/latt
Lattice constant for single cell.
layer_numb:
type: int, optional
argument path: init_surf_jdata/layer_numb
Number of atom layers constructing the slab.
z_min:
type: int, optional
argument path: init_surf_jdata/z_min
Thickness of slab without vacuum (Angstrom). If layer_numb is set, z_min will be ignored.
vacuum_max:
type: float
argument path: init_surf_jdata/vacuum_max
Maximal thickness of vacuum (Angstrom).
vacuum_min:
type: float, optional
argument path: init_surf_jdata/vacuum_min
Minimal thickness of vacuum (Angstrom). Default value is 2 times atomic radius.
vacuum_resol:
type: list[float]
argument path: init_surf_jdata/vacuum_resol
Interval of thickness of vacuum. If size of vacuum_resol is 1, the interval is fixed to its value. If size of vacuum_resol is 2, the interval is vacuum_resol[0] before mid_point, otherwise vacuum_resol[1] after mid_point.
vacuum_numb:
type: int, optional
argument path: init_surf_jdata/vacuum_numb
The total number of vacuum layers Necessary if vacuum_resol is empty.
mid_point:
type: float, optional
argument path: init_surf_jdata/mid_point
The mid point separating head region and tail region. Necessary if the size of vacuum_resol is 2 or 0.
head_ratio:
type: float, optional
argument path: init_surf_jdata/head_ratio
Ratio of vacuum layers in the nearby region with denser intervals(head region). Necessary if vacuum_resol is empty.
millers:
type: list[list[int]]
argument path: init_surf_jdata/millers
Miller indices.
relax_incar:
type: str, optional
argument path: init_surf_jdata/relax_incar
Path of INCAR for relaxation in VASP. Necessary if stages include 1.
scale:
type: list[float]
argument path: init_surf_jdata/scale
Scales for isotropic transforming cells.
skip_relax:
type: bool
argument path: init_surf_jdata/skip_relax
If it’s true, you may directly run stage 2 (perturb and scale) using an unrelaxed POSCAR.
pert_numb:
type: int
argument path: init_surf_jdata/pert_numb
Number of perturbations for each scaled (key scale) POSCAR.
pert_box:
type: float
argument path: init_surf_jdata/pert_box
Anisotropic Perturbation for cells (independent changes of lengths of three box vectors as well as angel among) in decimal formats. 9 elements of the 3x3 perturbation matrix will be randomly sampled from a uniform distribution (default) in the range [-pert_box, pert_box]. Such a perturbation matrix adds the identity matrix gives the actual transformation matrix for this perturbation operation.
pert_atom:
type: float
argument path: init_surf_jdata/pert_atom
Perturbation of atom coordinates (Angstrom). Random perturbations are performed on three coordinates of each atom by adding values randomly sampled from a uniform distribution in the range [-pert_atom, pert_atom].
coll_ndata:
type: int
argument path: init_surf_jdata/coll_ndata
Maximal number of collected data.

dpgen init_surf machine parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

init_surf_mdata:

type: dict
argument path: init_surf_mdata
machine.json file
api_version:
type: str, optional, default: 1.0
argument path: init_surf_mdata/api_version
Please set to 1.0
deepmd_version:
type: str, optional, default: 2
argument path: init_surf_mdata/deepmd_version
DeePMD-kit version, e.g. 2.1.3
fp:
type: dict
argument path: init_surf_mdata/fp
Parameters of command, machine, and resources for fp
command:
type: str
argument path: init_surf_mdata/fp/command
Command of a program.
machine:
type: dict
argument path: init_surf_mdata/fp/machine
batch_type:
type: str
argument path: init_surf_mdata/fp/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: init_surf_mdata/fp/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: init_surf_mdata/fp/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: init_surf_mdata/fp/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: init_surf_mdata/fp/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: init_surf_mdata/fp/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: init_surf_mdata/fp/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: init_surf_mdata/fp/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: init_surf_mdata/fp/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: init_surf_mdata/fp/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
For detailed class="dargs argument"id=init-surf-mdata-fp-machine-bohriumcontext-remote-profile-input-data>
input_data:
type: dict
argument path: init_surf_mdata/fp/machine[BohriumContext]/remote_profile/input_data
Configuration of job

resources:

type: dict
argument path: init_surf_mdata/fp/resources
number_node:
type: int, optional, default: 1
argument path: init_surf_mdata/fp/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: init_surf_mdata/fp/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: init_surf_mdata/fp/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: init_surf_mdata/fp/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: init_surf_mdata/fp/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: init_surf_mdata/fp/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: init_surf_mdata/fp/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: init_surf_mdata/fp/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: init_surf_mdata/fp/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: init_surf_mdata/fp/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: init_surf_mdata/fp/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: init_surf_mdata/fp/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: init_surf_mdata/fp/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: init_surf_mdata/fp/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: init_surf_mdata/fp/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: init_surf_mdata/fp/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: init_surf_mdata/fp/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: init_surf_mdata/fp/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: init_surf_mdata/fp/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: init_surf_mdata/fp/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_surf_mdata/fp/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: init_surf_mdata/fp/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: init_surf_mdata/fp/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: init_surf_mdata/fp/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: init_surf_mdata/fp/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_surf_mdata/fp/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_surf_mdata/fp/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: init_surf_mdata/fp/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: init_surf_mdata/fp/resources[Shell]/kwargs
This field is empty for this batch.

user_forward_files:: type: list, optional
argument path: init_surf_mdata/fp/user_forward_files
Files to be forwarded to the remote machine.

user_backward_files:: type: list, optional
argument path: init_surf_mdata/fp/user_backward_files
Files to be backwarded from the remote machine.

init_reaction

dpgen init_reaction is a workflow to initilize data for reactive systems of small gas-phase molecules. The workflow was introduced in the “Initialization” section of Energy & Fuels, 2021, 35 (1), 762–769.

To start the workflow, one needs a box containing reactive systems. The following packages are required for each of the step:

Exploring: LAMMPS
Sampling: MDDatasetBuilder
Labeling: Gaussian

The Exploring step uses LAMMPS pair_style reaxff to run a short ReaxMD NVT MD simulation. In the Sampling step, molecular clusters are taken and k-means clustering algorithm is applied to remove the redundancy, which is described in Nature Communications, 11, 5713 (2020). The Labeling step calculates energies and forces using the Gaussian package.

An example of reaction.json is given below:

1{ "type_map": [ "H", "O" ], "reaxff": { "data": "data.hydrogen", "ff": "ffield.reax.cho", "control": "lmp_control", "temp": 3000, "tau_t": 100, "dt": 0.1, "nstep": 10000, "dump_freq": 100 }, "cutoff": 3.5, "dataset_size": 100, "qmkeywords": "b3lyp/6-31g** force Geom=PrintInputOrient" class=p>} parameters, see parametes and machine parameters.

The genereated data can be used to continue DP-GEN concurrent learning workflow. Read Energy & Fuels, 2021, 35 (1), 762–769 for details.

dpgen init_reaction parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

init_reaction_jdata:

type: dict
argument path: init_reaction_jdata
Generate initial data for reactive systems for small gas-phase molecules, from a ReaxFF NVT MD trajectory.
type_map:
type: list[str]
argument path: init_reaction_jdata/type_map
Type map, which should match types in the initial data. e.g. [“C”, “H”, “O”]
reaxff:
type: dict
argument path: init_reaction_jdata/reaxff
Parameters for ReaxFF NVT MD.
data:
type: str
argument path: init_reaction_jdata/reaxff/data
Path to initial LAMMPS data file. The atom_style should be charge.
ff:
type: str
argument path: init_reaction_jdata/reaxff/ff
Path to ReaxFF force field file. Available in the lammps/potentials directory.
control:
type: str
argument path: init_reaction_jdata/reaxff/control
Path to ReaxFF control file.
temp:
type: int | float
argument path: init_reaction_jdata/reaxff/temp
Target Temperature for the NVT MD simulation. Unit: K.
dt:
type: int | float
argument path: init_reaction_jdata/reaxff/dt
Real time for every time step. Unit: fs.
tau_t:
type: int | float
argument path: init_reaction_jdata/reaxff/tau_t
Time to determine how rapidly the temperature. Unit: fs.
dump_freq:
type: int
argument path: init_reaction_jdata/reaxff/dump_freq
Frequency of time steps to collect trajectory.
nstep:
type: int
argument path: init_reaction_jdata/reaxff/nstep
Total steps to run the ReaxFF MD simulation.
cutoff:
type: float
argument path: init_reaction_jdata/cutoff
Cutoff radius to take clusters from the trajectory. Note that only a complete molecule or free radical will be taken.
dataset_size:
type: int
argument path: init_reaction_jdata/dataset_size
Collected dataset size for each bond type.
qmkeywords:
type: str
argument path: init_reaction_jdata/qmkeywords
Gaussian keywords for first-principle calculations. e.g. force mn15/6-31g** Geom=PrintInputOrient. Note that “force” job is necessary to collect data. Geom=PrintInputOrient should be used when there are more than 50 atoms in a cluster.

dpgen init_reaction machine parameters

Note

One can load, modify, and export the input file by using our effective web-based tool DP-GUI online or hosted using the command line interface dpgen gui. All parameters below can be set in DP-GUI. By clicking “SAVE JSON”, one can download the input file.

init_reaction_mdata:

type: dict
argument path: init_reaction_mdata
machine.json file
api_version:
type: str, optional, default: 1.0
argument path: init_reaction_mdata/api_version
Please set to 1.0
deepmd_version:
type: str, optional, default: 2
argument path: init_reaction_mdata/deepmd_version
DeePMD-kit version, e.g. 2.1.3
reaxff:
type: dict
argument path: init_reaction_mdata/reaxff
Parameters of command, machine, and resources for reaxff
command:
type: str
argument path: init_reaction_mdata/reaxff/command
Command of a program.
machine:
type: dict
argument path: init_reaction_mdata/reaxff/machine
batch_type:
type: str
argument path: init_reaction_mdata/reaxff/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: init_reaction_mdata/reaxff/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: init_reaction_mdata/reaxff/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: init_reaction_mdata/reaxff/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: init_reaction_mdata/reaxff/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: init_reaction_mdata/reaxff/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/reaxff/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/reaxff/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/reaxff/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/reaxff/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
input_data:
type: dict
argument path: init_reaction_mdata/reaxff/machine[BohriumContext]/remote_profile/input_data
Configuration of job
resources:
type: dict
argument path: init_reaction_mdata/reaxff/resources
number_node:
type: int, optional, default: 1
argument path: init_reaction_mdata/reaxff/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: init_reaction_mdata/reaxff/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: init_reaction_mdata/reaxff/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: init_reaction_mdata/reaxff/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: init_reaction_mdata/reaxff/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: init_reaction_mdata/reaxff/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: init_reaction_mdata/reaxff/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: init_reaction_mdata/reaxff/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: init_reaction_mdata/reaxff/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: init_reaction_mdata/reaxff/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/reaxff/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: init_reaction_mdata/reaxff/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/reaxff/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/reaxff/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: init_reaction_mdata/reaxff/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/reaxff/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/reaxff/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: init_reaction_mdata/reaxff/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: init_reaction_mdata/reaxff/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/reaxff/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: init_reaction_mdata/reaxff/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: init_reaction_mdata/reaxff/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: init_reaction_mdata/reaxff/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: init_reaction_mdata/reaxff/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/reaxff/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/reaxff/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: init_reaction_mdata/reaxff/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/reaxff/resources[Shell]/kwargs
This field is empty for this batch.
user_forward_files:
type: list, optional
argument path: init_reaction_mdata/reaxff/user_forward_files
Files to be forwarded to the remote machine.
user_backward_files:
type: list, optional
argument path: init_reaction_mdata/reaxff/user_backward_files
Files to be backwarded from the remote machine.
build:
type: dict
argument path: init_reaction_mdata/build
Parameters of command, machine, and resources for build
command:
type: str
argument path: init_reaction_mdata/build/command
Command of a program.
machine:
type: dict
argument path: init_reaction_mdata/build/machine
batch_type:
type: str
argument path: init_reaction_mdata/build/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: init_reaction_mdata/build/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: init_reaction_mdata/build/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: init_reaction_mdata/build/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: init_reaction_mdata/build/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: init_reaction_mdata/build/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/build/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/build/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/build/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/build/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
input_data:
type: dict
argument path: init_reaction_mdata/build/machine[BohriumContext]/remote_profile/input_data
Configuration of job
resources:
type: dict
argument path: init_reaction_mdata/build/resources
number_node:
type: int, optional, default: 1
argument path: init_reaction_mdata/build/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: init_reaction_mdata/build/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: init_reaction_mdata/build/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: init_reaction_mdata/build/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: init_reaction_mdata/build/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: init_reaction_mdata/build/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: init_reaction_mdata/build/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: init_reaction_mdata/build/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: init_reaction_mdata/build/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: init_reaction_mdata/build/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: init_reaction_mdata/build/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/build/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: init_reaction_mdata/build/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/build/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/build/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: init_reaction_mdata/build/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/build/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/build/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: init_reaction_mdata/build/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: init_reaction_mdata/build/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/build/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: init_reaction_mdata/build/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: init_reaction_mdata/build/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: init_reaction_mdata/build/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: init_reaction_mdata/build/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/build/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/build/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: init_reaction_mdata/build/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/build/resources[Shell]/kwargs
This field is empty for this batch.
user_forward_files:
type: list, optional
argument path: init_reaction_mdata/build/user_forward_files
Files to be forwarded to the remote machine.
user_backward_files:
type: list, optional
argument path: init_reaction_mdata/build/user_backward_files
Files to be backwarded from the remote machine.
fp:
type: dict
argument path: init_reaction_mdata/fp
Parameters of command, machine, and resources for fp
command:
type: str
argument path: init_reaction_mdata/fp/command
Command of a program.
machine:
type: dict
argument path: init_reaction_mdata/fp/machine
batch_type:
type: str
argument path: init_reaction_mdata/fp/machine/batch_type
The batch job system type. Option: OpenAPI, DistributedShell, Fugaku, PBS, Torque, Bohrium, SlurmJobArray, Slurm, LSF, SGE, Shell
local_root:
type: str | NoneType
argument path: init_reaction_mdata/fp/machine/local_root
The dir where the tasks and relating files locate. Typically the project dir.
remote_root:
type: str | NoneType, optional
argument path: init_reaction_mdata/fp/machine/remote_root
The dir where the tasks are executed on the remote machine. Only needed when context is not lazy-local.
clean_asynchronously:
type: bool, optional, default: False
argument path: init_reaction_mdata/fp/machine/clean_asynchronously
Clean the remote directory asynchronously after the job finishes.
Depending on the value of context_type, different sub args are accepted.
context_type:
type: str (flag key)
argument path: init_reaction_mdata/fp/machine/context_type
possible choices: SSHContext, LazyLocalContext, OpenAPIContext, LocalContext, HDFSContext, BohriumContext
The connection used to remote machine. Option: HDFSContext, BohriumContext, SSHContext, LocalContext, OpenAPIContext, LazyLocalContext
When context_type is set to SSHContext (or its aliases sshcontext, SSH, ssh):
remote_profile:
type: dict
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile
The information used to maintain the connection with remote machine.
hostname:
type: str
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/hostname
hostname or ip of ssh connection.
username:
type: str
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/username
username of target linux system
password:
type: str, optional
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/password
(deprecated) password of linux system. Please use SSH keys instead to improve security.
port:
type: int, optional, default: 22
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/port
ssh connection port.
key_filename:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/key_filename
key filename used by ssh connection. If left None, find key in ~/.ssh or use password for login
passphrase:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/passphrase
passphrase of key used by ssh connection
timeout:
type: int, optional, default: 10
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/timeout
timeout of ssh connection
totp_secret:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/totp_secret
Time-based one time password secret. It should be a base32-encoded string extracted from the 2D code.
tar_compress:
type: bool, optional, default: True
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/tar_compress
The archive will be compressed in upload and download if it is True. If not, compression will be skipped.
look_for_keys:
type: bool, optional, default: True
argument path: init_reaction_mdata/fp/machine[SSHContext]/remote_profile/look_for_keys
enable searching for discoverable private key files in ~/.ssh/
When context_type is set to LazyLocalContext (or its aliases lazylocalcontext, LazyLocal, lazylocal):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/fp/machine[LazyLocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to OpenAPIContext (or its aliases openapicontext, OpenAPI, openapi):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/fp/machine[OpenAPIContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to LocalContext (or its aliases localcontext, Local, local):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/fp/machine[LocalContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to HDFSContext (or its aliases hdfscontext, HDFS, hdfs):
remote_profile:
type: dict, optional
argument path: init_reaction_mdata/fp/machine[HDFSContext]/remote_profile
The information used to maintain the connection with remote machine. This field is empty for this context.
When context_type is set to BohriumContext (or its aliases bohriumcontext, Bohrium, bohrium, DpCloudServerContext, dpcloudservercontext, DpCloudServer, dpcloudserver, LebesgueContext, lebesguecontext, Lebesgue, lebesgue):
remote_profile:
type: dict
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile
The information used to maintain the connection with remote machine.
email:
type: str, optional
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile/email
Email
password:
type: str, optional
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile/password
Password
program_id:
type: int, alias: project_id
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile/program_id
Program ID
retry_count:
type: NoneType | int, optional, default: 2
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile/retry_count
The retry count when a job is terminated
ignore_exit_code:
type: bool, optional, default: True
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile/ignore_exit_code
The job state will be marked as finished if the exit code is non-zero when set to True. Otherwise,
the job state will be designated as terminated.
keep_backup:
type: bool, optional
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile/keep_backup
keep download and upload zip
input_data:
type: dict
argument path: init_reaction_mdata/fp/machine[BohriumContext]/remote_profile/input_data
Configuration of job
resources:
type: dict
argument path: init_reaction_mdata/fp/resources
number_node:
type: int, optional, default: 1
argument path: init_reaction_mdata/fp/resources/number_node
The number of node need for each job
cpu_per_node:
type: int, optional, default: 1
argument path: init_reaction_mdata/fp/resources/cpu_per_node
cpu numbers of each node assigned to each job.
gpu_per_node:
type: int, optional, default: 0
argument path: init_reaction_mdata/fp/resources/gpu_per_node
gpu numbers of each node assigned to each job.
queue_name:
type: str, optional, default: (empty string)
argument path: init_reaction_mdata/fp/resources/queue_name
The queue name of batch job scheduler system.
group_size:
type: int
argument path: init_reaction_mdata/fp/resources/group_size
The number of tasks in a job. 0 means infinity.
custom_flags:
type: typing.List[str], optional
argument path: init_reaction_mdata/fp/resources/custom_flags
The extra lines pass to job submitting script header
strategy:
type: dict, optional
argument path: init_reaction_mdata/fp/resources/strategy
strategies we use to generation job submitting scripts.
if_cuda_multi_devices:
type: bool, optional, default: False
argument path: init_reaction_mdata/fp/resources/strategy/if_cuda_multi_devices
If there are multiple nvidia GPUS on the node, and we want to assign the tasks to different GPUS.If true, dpdispatcher will manually export environment variable CUDA_VISIBLE_DEVICES to different task.Usually, this option will be used with Task.task_need_resources variable simultaneously.
ratio_unfinished:
type: float, optional, default: 0.0
argument path: init_reaction_mdata/fp/resources/strategy/ratio_unfinished
The ratio of tasks that can be unfinished.
customized_script_header_template_file:
type: str, optional
argument path: init_reaction_mdata/fp/resources/strategy/customized_script_header_template_file
The customized template file to generate job submitting script header, which overrides the default file.
para_deg:
type: int, optional, default: 1
argument path: init_reaction_mdata/fp/resources/para_deg
Decide how many tasks will be run in parallel.
source_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/fp/resources/source_list
The env file to be sourced before the command execution.
module_purge:
type: bool, optional, default: False
argument path: init_reaction_mdata/fp/resources/module_purge
Remove all modules on HPC system before module load (module_list)
module_unload_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/fp/resources/module_unload_list
The modules to be unloaded on HPC system before submitting jobs
module_list:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/fp/resources/module_list
The modules to be loaded on HPC system before submitting jobs
envs:
type: dict, optional, default: {}
argument path: init_reaction_mdata/fp/resources/envs
The environment variables to be exported on before submitting jobs
prepend_script:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/fp/resources/prepend_script
Optional script run before jobs submitted.
append_script:
type: typing.List[str], optional, default: []
argument path: init_reaction_mdata/fp/resources/append_script
Optional script run after jobs submitted.
wait_time:
type: float | int, optional, default: 0
argument path: init_reaction_mdata/fp/resources/wait_time
The waitting time in second after a single task submitted
Depending on the value of batch_type, different sub args are accepted.
batch_type:
type: str (flag key)
argument path: init_reaction_mdata/fp/resources/batch_type
possible choices: Fugaku, Slurm, DistributedShell, Bohrium, LSF, SGE, OpenAPI, SlurmJobArray, Torque, PBS, Shell
The batch job system type loaded from machine/batch_type.
When batch_type is set to Fugaku (or its alias fugaku):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[Fugaku]/kwargs
This field is empty for this batch.
When batch_type is set to Slurm (or its alias slurm):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[Slurm]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/fp/resources[Slurm]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
When batch_type is set to DistributedShell (or its alias distributedshell):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[DistributedShell]/kwargs
This field is empty for this batch.
When batch_type is set to Bohrium (or its aliases bohrium, Lebesgue, lebesgue, DpCloudServer, dpcloudserver):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[Bohrium]/kwargs
This field is empty for this batch.
When batch_type is set to LSF (or its alias lsf):
kwargs:
type: dict
argument path: init_reaction_mdata/fp/resources[LSF]/kwargs
Extra arguments.
gpu_usage:
type: bool, optional, default: False
argument path: init_reaction_mdata/fp/resources[LSF]/kwargs/gpu_usage
Choosing if GPU is used in the calculation step.
gpu_new_syntax:
type: bool, optional, default: False
argument path: init_reaction_mdata/fp/resources[LSF]/kwargs/gpu_new_syntax
For LFS >= 10.1.0.3, new option -gpu for #BSUB could be used. If False, and old syntax would be used.
gpu_exclusive:
type: bool, optional, default: True
argument path: init_reaction_mdata/fp/resources[LSF]/kwargs/gpu_exclusive
Only take effect when new syntax enabled. Control whether submit tasks in exclusive way for GPU.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/fp/resources[LSF]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #BSUB
When batch_type is set to SGE (or its alias sge):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[SGE]/kwargs
This field is empty for this batch.
When batch_type is set to OpenAPI (or its alias openapi):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[OpenAPI]/kwargs
This field is empty for this batch.
When batch_type is set to SlurmJobArray (or its alias slurmjobarray):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[SlurmJobArray]/kwargs
Extra arguments.
custom_gpu_line:
type: str | NoneType, optional, default: None
argument path: init_reaction_mdata/fp/resources[SlurmJobArray]/kwargs/custom_gpu_line
Custom GPU configuration, starting with #SBATCH
slurm_job_size:
type: int, optional, default: 1
argument path: init_reaction_mdata/fp/resources[SlurmJobArray]/kwargs/slurm_job_size
Number of tasks in a Slurm job
When batch_type is set to Torque (or its alias torque):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[Torque]/kwargs
This field is empty for this batch.
When batch_type is set to PBS (or its alias pbs):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[PBS]/kwargs
This field is empty for this batch.
When batch_type is set to Shell (or its alias shell):
kwargs:
type: dict, optional
argument path: init_reaction_mdata/fp/resources[Shell]/kwargs
This field is empty for this batch.
user_forward_files:
type: list, optional
argument path: init_reaction_mdata/fp/user_forward_files
Files to be forwarded to the remote machine.
user_backward_files:
type: list, optional
argument path: init_reaction_mdata/fp/user_backward_files
Files to be backwarded from the remote machine.

Key words	data structure	example	description
structures	List of String	[“confs/std-*”]	path of different structures
interaction	Dict	See above	description of the task type and atomic interaction
type	String	“vasp”	task type
incar	String	“vasp_input/INCAR”	path for INCAR file in vasp
potcar_prefix	String	“vasp_input”	prefix of path for POTCAR file in vasp, default = “”
potcars	Dict	{“Al”: “POTCAR.al”}	key is element type and value is potcar name
relaxation	Dict	See above	calculation type and setting for relaxation
cal_type	String	“relaxation” or “static”	calculation type
cal_setting	Dict	See above	calculation setting
relax_pos	Boolean	true	relax atomic position or not, default = true for relaxation
relax_shape	Boolean	true	relax box shape or not, default = true for relaxation
relax_vol	Boolean	true	relax box volume or not, default = true for relaxation
ediff	Float	1e-6	set `EDIFF` parameter in INCAR files
ediffg	Float	-0.01	set `EDIFFG` parameter in INCAR files
encut	Int	650	set `encut` parameter in INCAR files
kspacing	Float	0.1	set `KSPACING` parameter in INCAR files
kgamma	Boolean	false	set `KGAMMA` parameter in INCAR files

Key words	data structure	example	description
model	String or List of String	“frozen_model.pb”	model file for atomic interaction
in_lammps	String	“lammps_input/in.lammps”	input file for lammps commands
type_map	Dict	{“Al”: 0}	key is element type and value is type number. DP starts from 0, others starts from 1
etol	Float	0	stopping tolerance for energy
ftol	Float	1e-10	stopping tolerance for force
maxiter	Int	5000	max iterations of minimizer
maxeval	Int	500000	max number of force/energy evaluations

Key words	data structure	example	description
type	String	“eos”	property type
skip	Boolean	true	whether to skip current property or not
start_confs_path	String	“../vasp/confs”	start from the equilibrium configuration in other path only for the current property type
cal_setting[“input_prop”]	String	“lammps_input/lammps_high”	input commands file
cal_setting[“overwrite_interaction”]	Dict		overwrite the interaction in the `interaction` part only for the current property type

Key words	data structure	example	description
vol_start	Float	0.9	the starting volume related to the equilibrium structure
vol_end	Float	1.1	the biggest volume related to the equilibrium structure
vol_step	Float	0.01	the volume increment related to the equilibrium structure
vol_abs	Boolean	false	whether to treat vol_start, vol_end and vol_step as absolute volume or not (as relative volume), default = false

Key words	data structure	example	description
norm_deform	Float	1e-2	deformation in xx, yy, zz, default = 1e-2
shear_deform	Float	1e-2	deformation in other directions, default = 1e-2

`compute`(output_file, print_file, path_to_work)	Postprocess the finished tasks to compute the property.
`make_confs`(path_to_work, path_to_equi[, refine])	Make configurations needed to compute the property.
`post_process`(task_list)	post_process the KPOINTS file in elastic.
`task_param`()	Return the parameter of each computational task, for example, {'ediffg': 1e-4}.
`task_type`()	Return the type of each computational task, for example, 'relaxation', 'static'....

`compute`(output_file, print_file, path_to_work)	Postprocess the finished tasks to compute the property.
`make_confs`(path_to_work, path_to_equi[, refine])	Make configurations needed to compute the property.
`post_process`(task_list)	post_process the KPOINTS file in elastic.
`task_param`()	Return the parameter of each computational task, for example, {'ediffg': 1e-4}.
`task_type`()	Return the type of each computational task, for example, 'relaxation', 'static'....

`compute`(output_file, print_file, path_to_work)	Postprocess the finished tasks to compute the property.
`make_confs`(path_to_work, path_to_equi[, refine])	Make configurations needed to compute the property.
`post_process`(task_list)	post_process the KPOINTS file in elastic.
`task_param`()	Return the parameter of each computational task, for example, {'ediffg': 1e-4}.
`task_type`()	Return the type of each computational task, for example, 'relaxation', 'static'....

centralize_slab
return_direction

DPGEN’s documentation

Overview

About DP-GEN

Highlighted features

Download and install

Use DP-GEN

Case Studies

License

Command line interface

Sub-commands

init_surf

Positional Arguments

init_bulk

Positional Arguments

auto_gen_param

Positional Arguments

init_reaction

Positional Arguments

run

Positional Arguments

Named Arguments

run/report

Positional Arguments

Named Arguments

collect

Positional Arguments

Named Arguments

simplify

Positional Arguments

Named Arguments

autotest

Positional Arguments

Named Arguments

db

Positional Arguments

gui

Named Arguments

Code Structure

Run

Overview of the Run process

Example-of-param.json

basics

data

training

exploration

labeling

Example of machine.json

DPDispatcher Update Note

New DPDispatcher

Performing train task at a local workstation

Perform model_devi task at a local Slurm cluster

Perform fp task in a remote PBS cluster

dpgen run param parameters

dpgen run machine parameters

Init

Init_bulk

dpgen init_bulk parameters

dpgen init_bulk machine parameters

Init_surf

dpgen init_surf parameters

dpgen init_surf machine parameters

init_reaction

dpgen init_reaction parameters

dpgen init_reaction machine parameters

Simplify

Simplify

dpgen simplify parameters

dpgen simplify machine parameters

Auto test

Autotest Overview: Autotest for Deep Generator

Relaxation

Task type

Property type

Make run and post

Make

Run

Post

Relaxation

Relaxation get started and input examples

Relaxation make

Pull requests - How to contribute 

dargs.dargs.ArgumentKeyError: [at location `xxx`] undefined key xxx is not allowed in strict mode.