deepmd_utils.utils package
Submodules
deepmd_utils.utils.argcheck module
- class deepmd_utils.utils.argcheck.ArgsPlugin[source]
Bases:
object
Methods
get_all_argument
([exclude_hybrid])Get all arguments.
register
(name[, alias])Register a descriptor argument plugin.
- deepmd_utils.utils.argcheck.descrpt_variant_type_args(exclude_hybrid: bool = False) Variant [source]
- deepmd_utils.utils.argcheck.normalize_fitting_weight(fitting_keys, data_keys, fitting_weight=None)[source]
deepmd_utils.utils.argcheck_nvnmd module
deepmd_utils.utils.batch_size module
- class deepmd_utils.utils.batch_size.AutoBatchSize(initial_batch_size: int = 1024, factor: float = 2.0)[source]
Bases:
ABC
This class allows DeePMD-kit to automatically decide the maximum batch size that will not cause an OOM error.
- Parameters
Notes
In some CPU environments, the program may be directly killed when OOM. In this case, by default the batch size will not be increased for CPUs. The environment variable DP_INFER_BATCH_SIZE can be set as the batch size.
In other cases, we assume all OOM error will raise
OutOfMemoryError
.- Attributes
Methods
execute
(callable, start_index, natoms)Excuate a method with given batch size.
execute_all
(callable, total_size, natoms, ...)Excuate a method with all given data.
Check if GPU is available.
is_oom_error
(e)Check if the exception is an OOM error.
- execute(callable: Callable, start_index: int, natoms: int) Tuple[int, tuple] [source]
Excuate a method with given batch size.
- Parameters
- Returns
- Raises
OutOfMemoryError
OOM when batch size is 1
- execute_all(callable: Callable, total_size: int, natoms: int, *args, **kwargs) Tuple[ndarray] [source]
Excuate a method with all given data.
deepmd_utils.utils.compat module
Module providing compatibility between 0.x.x and 1.x.x input versions.
- deepmd_utils.utils.compat.convert_input_v0_v1(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any] [source]
Convert input from v0 format to v1.
- deepmd_utils.utils.compat.convert_input_v1_v2(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any] [source]
- deepmd_utils.utils.compat.deprecate_numb_test(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any] [source]
Deprecate numb_test since v2.1. It has taken no effect since v2.0.
See #1243.
deepmd_utils.utils.data module
- class deepmd_utils.utils.data.DeepmdData(sys_path: str, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set: bool = False, sort_atoms: bool = True)[source]
Bases:
object
Class for a data system.
It loads data from hard disk, and mantains the data as a data_dict
- Parameters
- sys_path
Path to the data system
- set_prefix
Prefix for the directories of different sets
- shuffle_test
If the test data are shuffled
- type_map
Gives the name of different atom types
- optional_type_map
If the type_map.raw in each system is optional
- modifier
Data modifier that has the method modify_data
- trn_all_set
Use all sets as training dataset. Otherwise, if the number of sets is more than 1, the last set is left for test.
- sort_atomsbool
Sort atoms by atom types. Required to enable when the data is directly feeded to descriptors except mixed types.
Methods
add
(key, ndof[, atomic, must, high_prec, ...])Add a data item that to be loaded.
avg
(key)Return the average value of an item.
check_batch_size
(batch_size)Check if the system can get a batch of data with batch_size frames.
check_test_size
(test_size)Check if the system can get a test dataset with test_size frames.
Get atom types.
get_batch
(batch_size)Get a batch of data with batch_size frames.
Get the data_dict.
Get number of atoms.
get_natoms_vec
(ntypes)Get number of atoms and number of atoms in different types.
Number of atom types in the system.
get_numb_batch
(batch_size, set_idx)Get the number of batches in a set.
Get number of training sets.
get_sys_numb_batch
(batch_size)Get the number of batches in the data system.
get_test
([ntests])Get the test data with ntests frames.
Get the type map.
reduce
(key_out, key_in)Generate a new item from the reduction of another atom.
reset_get_batch
- add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None)[source]
Add a data item that to be loaded.
- Parameters
- key
The key of the item. The corresponding data is stored in sys_path/set.*/key.npy
- ndof
The number of dof
- atomic
The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof
- must
The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0
- high_prec
Load the data and store in float64, otherwise in float32
- type_sel
Select certain type of atoms
- repeat
The data will be repeated repeat times.
- default
float
, default=0. default value of data
- dtype
np.dtype
,optional
the dtype of data, overwrites high_prec if provided
- check_batch_size(batch_size)[source]
Check if the system can get a batch of data with batch_size frames.
- check_test_size(test_size)[source]
Check if the system can get a test dataset with test_size frames.
- get_batch(batch_size: int) dict [source]
Get a batch of data with batch_size frames. The frames are randomly picked from the data system.
- Parameters
- batch_size
size of the batch
- get_natoms_vec(ntypes: int)[source]
Get number of atoms and number of atoms in different types.
- Parameters
- ntypes
Number of types (may be larger than the actual number of types in the system).
- Returns
natoms
natoms[0]: number of local atoms natoms[1]: total number of atoms held by this processor natoms[i]: 2 <= i < Ntypes+2, number of type i atoms
- get_test(ntests: int = -1) dict [source]
Get the test data with ntests frames.
- Parameters
- ntests
Size of the test data set. If ntests is -1, all test data will be get.
deepmd_utils.utils.data_system module
- class deepmd_utils.utils.data_system.DeepmdDataSystem(systems: List[str], batch_size: int, test_size: int, rcut: Optional[float] = None, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set=False, sys_probs=None, auto_prob_style='prob_sys_size', sort_atoms: bool = True)[source]
Bases:
object
Class for manipulating many data systems.
It is implemented with the help of DeepmdData
- Attributes
default_mesh
Mesh for each system.
Methods
add
(key, ndof[, atomic, must, high_prec, ...])Add a data item that to be loaded.
add_dict
(adict)Add items to the data system by a dict.
get_batch
([sys_idx])Get a batch of data from the data systems.
Get a batch of data from the data systems in the mixed way.
Get the batch size.
get_batch_standard
([sys_idx])Get a batch of data from the data systems in the standard way.
Get the total number of batches.
Get the number of data systems.
Get the number of types.
get_sys
(idx)Get a certain data system.
get_sys_ntest
([sys_idx])Get number of tests for the currently selected system, or one defined by sys_idx.
get_test
([sys_idx, n_test])Get test data from the the data systems.
Get the type map.
reduce
(key_out, key_in)Generate a new item from the reduction of another atom.
compute_energy_shift
get_data_dict
print_summary
set_sys_probs
- add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0)[source]
Add a data item that to be loaded.
- Parameters
- key
The key of the item. The corresponding data is stored in sys_path/set.*/key.npy
- ndof
The number of dof
- atomic
The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof
- must
The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0
- high_prec
Load the data and store in float64, otherwise in float32
- type_sel
Select certain type of atoms
- repeat
The data will be repeated repeat times.
- default, default=0.
Default value of data
- add_dict(adict: dict) None [source]
Add items to the data system by a dict. adict should have items like .. code-block:: python.
- adict[key] = {
“ndof”: ndof, “atomic”: atomic, “must”: must, “high_prec”: high_prec, “type_sel”: type_sel, “repeat”: repeat,
}
For the explaination of the keys see add
- get_batch(sys_idx: Optional[int] = None) dict [source]
Get a batch of data from the data systems.
- Parameters
- sys_idx
int
The index of system from which the batch is get. If sys_idx is not None, sys_probs and auto_prob_style are ignored If sys_idx is None, automatically determine the system according to sys_probs or auto_prob_style, see the following. This option does not work for mixed systems.
- sys_idx
- Returns
dict
The batch data
- get_batch_mixed() dict [source]
Get a batch of data from the data systems in the mixed way.
- Returns
dict
The batch data
- get_batch_standard(sys_idx: Optional[int] = None) dict [source]
Get a batch of data from the data systems in the standard way.
- get_sys(idx: int) DeepmdData [source]
Get a certain data system.
- get_sys_ntest(sys_idx=None)[source]
Get number of tests for the currently selected system, or one defined by sys_idx.
- get_test(sys_idx: Optional[int] = None, n_test: int = -1)[source]
Get test data from the the data systems.
- Parameters
- sys_idx
The test dat of system with index sys_idx will be returned. If is None, the currently selected system will be returned.
- n_test
Number of test data. If set to -1 all test data will be get.
deepmd_utils.utils.errors module
deepmd_utils.utils.model_stat module
- deepmd_utils.utils.model_stat.make_stat_input(data, nbatches, merge_sys=True)[source]
Pack data for statistics.
- Parameters
- Returns
- all_stat:
A dictionary of list of list storing data for stat. if merge_sys == False data can be accessed by
all_stat[key][sys_idx][batch_idx][frame_idx]
- else merge_sys == True can be accessed by
all_stat[key][batch_idx][frame_idx]
deepmd_utils.utils.pair_tab module
- class deepmd_utils.utils.pair_tab.PairTab(filename: str)[source]
Bases:
object
Pairwise tabulated potential.
- Parameters
- filename
File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.
Methods
get
()Get the serialized table.
reinit
(filename)Initialize the tabulated interaction.
- reinit(filename: str) None [source]
Initialize the tabulated interaction.
- Parameters
- filename
File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.
deepmd_utils.utils.path module
- class deepmd_utils.utils.path.DPH5Path(path: str)[source]
Bases:
DPPath
The path class to data system (DeepmdData) for HDF5 files.
- Parameters
- path
str
path
- path
Notes
- OS - HDF5 relationship:
directory - Group file - Dataset
Methods
glob
(pattern)Search path using the glob pattern.
is_dir
()Check if self is directory.
is_file
()Check if self is file.
Load NumPy array.
load_txt
([dtype])Load NumPy array from text.
rglob
(pattern)This is like calling
DPPath.glob()
with **/ added in front of the given relative pattern.- load_numpy() ndarray [source]
Load NumPy array.
- Returns
np.ndarray
loaded NumPy array
- class deepmd_utils.utils.path.DPOSPath(path: str)[source]
Bases:
DPPath
The OS path class to data system (DeepmdData) for real directories.
- Parameters
- path
str
path
- path
Methods
glob
(pattern)Search path using the glob pattern.
is_dir
()Check if self is directory.
is_file
()Check if self is file.
Load NumPy array.
load_txt
(**kwargs)Load NumPy array from text.
rglob
(pattern)This is like calling
DPPath.glob()
with **/ added in front of the given relative pattern.- load_numpy() ndarray [source]
Load NumPy array.
- Returns
np.ndarray
loaded NumPy array
- load_txt(**kwargs) ndarray [source]
Load NumPy array from text.
- Returns
np.ndarray
loaded NumPy array
- class deepmd_utils.utils.path.DPPath(path: str)[source]
Bases:
ABC
The path class to data system (DeepmdData).
- Parameters
- path
str
path
- path
Methods
glob
(pattern)Search path using the glob pattern.
is_dir
()Check if self is directory.
is_file
()Check if self is file.
Load NumPy array.
load_txt
(**kwargs)Load NumPy array from text.
rglob
(pattern)This is like calling
DPPath.glob()
with **/ added in front of the given relative pattern.- abstract load_numpy() ndarray [source]
Load NumPy array.
- Returns
np.ndarray
loaded NumPy array
- abstract load_txt(**kwargs) ndarray [source]
Load NumPy array from text.
- Returns
np.ndarray
loaded NumPy array
deepmd_utils.utils.plugin module
Base of plugin systems.
- class deepmd_utils.utils.plugin.Plugin[source]
Bases:
object
A class to register and restore plugins.
Examples
>>> plugin = Plugin() >>> @plugin.register("xx") def xxx(): pass >>> print(plugin.plugins['xx'])
Methods
get_plugin
(key)Visit a plugin by key.
register
(key)Register a plugin.
- class deepmd_utils.utils.plugin.PluginVariant(*args, **kwargs)[source]
Bases:
object
A class to remove type from input arguments.
- class deepmd_utils.utils.plugin.VariantABCMeta(name, bases, namespace, /, **kwargs)[source]
Bases:
VariantMeta
,ABCMeta
Methods
__call__
(*args, **kwargs)Remove type and keys that starts with underline.
mro
(/)Return a type's method resolution order.
register
(subclass)Register a virtual subclass of an ABC.
deepmd_utils.utils.random module
- deepmd_utils.utils.random.choice(a: ndarray, p: Optional[ndarray] = None)[source]
Generates a random sample from a given 1-D array.
- Parameters
- a
np.ndarray
A random sample is generated from its elements.
- p
np.ndarray
The probabilities associated with each entry in a.
- a
- Returns
np.ndarray
arrays with results and their shapes
- deepmd_utils.utils.random.random(size=None)[source]
Return random floats in the half-open interval [0.0, 1.0).
- Parameters
- size
Output shape.
- Returns
np.ndarray
Arrays with results and their shapes.
- deepmd_utils.utils.random.seed(val: Optional[int] = None)[source]
Seed the generator.
- Parameters
- val
int
Seed.
- val
- deepmd_utils.utils.random.shuffle(x: ndarray)[source]
Modify a sequence in-place by shuffling its contents.
- Parameters
- x
np.ndarray
The array or list to be shuffled.
- x