deepmd.tf.utils package
- class deepmd.tf.utils.DeepmdData(sys_path: str, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set: bool = False, sort_atoms: bool = True)[source]
Bases:
objectClass for a data system.
It loads data from hard disk, and mantains the data as a data_dict
- Parameters
- sys_path
Path to the data system
- set_prefix
Prefix for the directories of different sets
- shuffle_test
If the test data are shuffled
- type_map
Gives the name of different atom types
- optional_type_map
If the type_map.raw in each system is optional
- modifier
Data modifier that has the method modify_data
- trn_all_set
Use all sets as training dataset. Otherwise, if the number of sets is more than 1, the last set is left for test.
- sort_atomsbool
Sort atoms by atom types. Required to enable when the data is directly feeded to descriptors except mixed types.
Methods
add(key, ndof[, atomic, must, high_prec, ...])Add a data item that to be loaded.
avg(key)Return the average value of an item.
check_batch_size(batch_size)Check if the system can get a batch of data with batch_size frames.
check_test_size(test_size)Check if the system can get a test dataset with test_size frames.
Get atom types.
get_batch(batch_size)Get a batch of data with batch_size frames.
Get the data_dict.
get_item_torch(index)Get a single frame data .
Get number of atoms.
get_natoms_vec(ntypes)Get number of atoms and number of atoms in different types.
Number of atom types in the system.
get_numb_batch(batch_size, set_idx)Get the number of batches in a set.
Get number of training sets.
get_sys_numb_batch(batch_size)Get the number of batches in the data system.
get_test([ntests])Get the test data with ntests frames.
Get the type map.
reduce(key_out, key_in)Generate a new item from the reduction of another atom.
reformat_data_torch(data)Modify the data format for the requirements of Torch backend.
reset_get_batch
- add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None, output_natoms_for_type_sel: bool = False)[source]
Add a data item that to be loaded.
- Parameters
- key
The key of the item. The corresponding data is stored in sys_path/set.*/key.npy
- ndof
The number of dof
- atomic
The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof
- must
The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0
- high_prec
Load the data and store in float64, otherwise in float32
- type_sel
Select certain type of atoms
- repeat
The data will be repeated repeat times.
- default
float, default=0. default value of data
- dtype
np.dtype,optional the dtype of data, overwrites high_prec if provided
- output_natoms_for_type_selbool,
optional if True and type_sel is True, the atomic dimension will be natoms instead of nsel
- check_batch_size(batch_size)[source]
Check if the system can get a batch of data with batch_size frames.
- check_test_size(test_size)[source]
Check if the system can get a test dataset with test_size frames.
- get_batch(batch_size: int) dict[source]
Get a batch of data with batch_size frames. The frames are randomly picked from the data system.
- Parameters
- batch_size
size of the batch
- get_item_torch(index: int) dict[source]
Get a single frame data . The frame is picked from the data system by index. The index is coded across all the sets.
- Parameters
- index
index of the frame
- get_natoms_vec(ntypes: int)[source]
Get number of atoms and number of atoms in different types.
- Parameters
- ntypes
Number of types (may be larger than the actual number of types in the system).
- Returns
natomsnatoms[0]: number of local atoms natoms[1]: total number of atoms held by this processor natoms[i]: 2 <= i < Ntypes+2, number of type i atoms
- get_test(ntests: int = -1) dict[source]
Get the test data with ntests frames.
- Parameters
- ntests
Size of the test data set. If ntests is -1, all test data will be get.
- reduce(key_out: str, key_in: str)[source]
Generate a new item from the reduction of another atom.
- Parameters
- key_out
The name of the reduced item
- key_in
The name of the data item to be reduced
- class deepmd.tf.utils.DeepmdDataSystem(systems: List[str], batch_size: int, test_size: int, rcut: Optional[float] = None, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set=False, sys_probs=None, auto_prob_style='prob_sys_size', sort_atoms: bool = True)[source]
Bases:
objectClass for manipulating many data systems.
It is implemented with the help of DeepmdData
- Attributes
default_meshMesh for each system.
Methods
add(key, ndof[, atomic, must, high_prec, ...])Add a data item that to be loaded.
add_dict(adict)Add items to the data system by a dict.
get_batch([sys_idx])Get a batch of data from the data systems.
Get a batch of data from the data systems in the mixed way.
Get the batch size.
get_batch_standard([sys_idx])Get a batch of data from the data systems in the standard way.
Get the total number of batches.
Get the number of data systems.
Get the number of types.
get_sys(idx)Get a certain data system.
get_sys_ntest([sys_idx])Get number of tests for the currently selected system, or one defined by sys_idx.
get_test([sys_idx, n_test])Get test data from the the data systems.
Get the type map.
reduce(key_out, key_in)Generate a new item from the reduction of another atom.
compute_energy_shift
get_data_dict
print_summary
set_sys_probs
- add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None, output_natoms_for_type_sel: bool = False)[source]
Add a data item that to be loaded.
- Parameters
- key
The key of the item. The corresponding data is stored in sys_path/set.*/key.npy
- ndof
The number of dof
- atomic
The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof
- must
The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0
- high_prec
Load the data and store in float64, otherwise in float32
- type_sel
Select certain type of atoms
- repeat
The data will be repeated repeat times.
- default, default=0.
Default value of data
- dtype
The dtype of data, overwrites high_prec if provided
- output_natoms_for_type_selbool
If True and type_sel is True, the atomic dimension will be natoms instead of nsel
- add_dict(adict: dict) None[source]
Add items to the data system by a dict. adict should have items like .. code-block:: python.
- adict[key] = {
“ndof”: ndof, “atomic”: atomic, “must”: must, “high_prec”: high_prec, “type_sel”: type_sel, “repeat”: repeat,
}
For the explaination of the keys see add
- get_batch(sys_idx: Optional[int] = None) dict[source]
Get a batch of data from the data systems.
- Parameters
- sys_idx
int The index of system from which the batch is get. If sys_idx is not None, sys_probs and auto_prob_style are ignored If sys_idx is None, automatically determine the system according to sys_probs or auto_prob_style, see the following. This option does not work for mixed systems.
- sys_idx
- Returns
dictThe batch data
- get_batch_mixed() dict[source]
Get a batch of data from the data systems in the mixed way.
- Returns
dictThe batch data
- get_batch_standard(sys_idx: Optional[int] = None) dict[source]
Get a batch of data from the data systems in the standard way.
- get_sys(idx: int) DeepmdData[source]
Get a certain data system.
- get_sys_ntest(sys_idx=None)[source]
Get number of tests for the currently selected system, or one defined by sys_idx.
- get_test(sys_idx: Optional[int] = None, n_test: int = -1)[source]
Get test data from the the data systems.
- Parameters
- sys_idx
The test dat of system with index sys_idx will be returned. If is None, the currently selected system will be returned.
- n_test
Number of test data. If set to -1 all test data will be get.
- class deepmd.tf.utils.LearningRateExp(start_lr: float, stop_lr: float = 5e-08, decay_steps: int = 5000, decay_rate: float = 0.95)[source]
Bases:
objectThe exponentially decaying learning rate.
The learning rate at step \(t\) is given by
\[\alpha(t) = \alpha_0 \lambda ^ { t / \tau }\]where \(\alpha\) is the learning rate, \(\alpha_0\) is the starting learning rate, \(\lambda\) is the decay rate, and \(\tau\) is the decay steps.
- Parameters
- start_lr
Starting learning rate \(\alpha_0\)
- stop_lr
Stop learning rate \(\alpha_1\)
- decay_steps
Learning rate decay every this number of steps \(\tau\)
- decay_rate
The decay rate \(\lambda\). If stop_step is provided in build, then it will be determined automatically and overwritten.
Methods
build(global_step[, stop_step])Build the learning rate.
start_lr()Get the start lr.
value(step)Get the lr at a certain step.
- build(global_step: Tensor, stop_step: Optional[int] = None) Tensor[source]
Build the learning rate.
- Parameters
- global_step
The tf Tensor prividing the global training step
- stop_step
The stop step. If provided, the decay_rate will be determined automatically and overwritten.
- Returns
learning_rateThe learning rate
- class deepmd.tf.utils.PairTab(filename: str, rcut: Optional[float] = None)[source]
Bases:
objectPairwise tabulated potential.
- Parameters
- filename
File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.
Methods
get()Get the serialized table.
reinit(filename[, rcut])Initialize the tabulated interaction.
deserialize
serialize
- reinit(filename: str, rcut: Optional[float] = None) None[source]
Initialize the tabulated interaction.
- Parameters
- filename
File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.
- class deepmd.tf.utils.Plugin[source]
Bases:
objectA class to register and restore plugins.
Examples
>>> plugin = Plugin() >>> @plugin.register("xx") def xxx(): pass >>> print(plugin.plugins["xx"])
Methods
get_plugin(key)Visit a plugin by key.
register(key)Register a plugin.
- class deepmd.tf.utils.PluginVariant(*args, **kwargs)[source]
Bases:
objectA class to remove type from input arguments.
Submodules
deepmd.tf.utils.argcheck module
Alias for backward compatibility.
deepmd.tf.utils.batch_size module
- class deepmd.tf.utils.batch_size.AutoBatchSize(initial_batch_size: int = 1024, factor: float = 2.0)[source]
Bases:
AutoBatchSizeMethods
execute(callable, start_index, natoms)Excuate a method with given batch size.
execute_all(callable, total_size, natoms, ...)Excuate a method with all given data.
Check if GPU is available.
is_oom_error(e)Check if the exception is an OOM error.
deepmd.tf.utils.compat module
Alias for backward compatibility.
- deepmd.tf.utils.compat.convert_input_v0_v1(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any][source]
Convert input from v0 format to v1.
- deepmd.tf.utils.compat.convert_input_v1_v2(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any][source]
deepmd.tf.utils.compress module
deepmd.tf.utils.convert module
- deepmd.tf.utils.convert.convert_012_to_21(input_model: str, output_model: str)[source]
Convert DP 0.12 graph to 2.1 graph.
- deepmd.tf.utils.convert.convert_10_to_21(input_model: str, output_model: str)[source]
Convert DP 1.0 graph to 2.1 graph.
- deepmd.tf.utils.convert.convert_12_to_21(input_model: str, output_model: str)[source]
Convert DP 1.2 graph to 2.1 graph.
- deepmd.tf.utils.convert.convert_13_to_21(input_model: str, output_model: str)[source]
Convert DP 1.3 graph to 2.1 graph.
- deepmd.tf.utils.convert.convert_20_to_21(input_model: str, output_model: str)[source]
Convert DP 2.0 graph to 2.1 graph.
- deepmd.tf.utils.convert.convert_dp012_to_dp10(file: str)[source]
Convert DP 0.12 graph text to 1.0 graph text.
- Parameters
- file
str filename of the graph text
- file
- deepmd.tf.utils.convert.convert_dp10_to_dp11(file: str)[source]
Convert DP 1.0 graph text to 1.1 graph text.
- Parameters
- file
str filename of the graph text
- file
- deepmd.tf.utils.convert.convert_dp12_to_dp13(file: str)[source]
Convert DP 1.2 graph text to 1.3 graph text.
- Parameters
- file
str filename of the graph text
- file
- deepmd.tf.utils.convert.convert_dp13_to_dp20(fname: str)[source]
Convert DP 1.3 graph text to 2.0 graph text.
- Parameters
- fname
str filename of the graph text
- fname
- deepmd.tf.utils.convert.convert_pb_to_pbtxt(pbfile: str, pbtxtfile: str)[source]
Convert DP graph to graph text.
- deepmd.tf.utils.convert.convert_pbtxt_to_pb(pbtxtfile: str, pbfile: str)[source]
Convert DP graph text to graph.
deepmd.tf.utils.data module
Alias for backward compatibility.
- class deepmd.tf.utils.data.DeepmdData(sys_path: str, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set: bool = False, sort_atoms: bool = True)[source]
Bases:
objectClass for a data system.
It loads data from hard disk, and mantains the data as a data_dict
- Parameters
- sys_path
Path to the data system
- set_prefix
Prefix for the directories of different sets
- shuffle_test
If the test data are shuffled
- type_map
Gives the name of different atom types
- optional_type_map
If the type_map.raw in each system is optional
- modifier
Data modifier that has the method modify_data
- trn_all_set
Use all sets as training dataset. Otherwise, if the number of sets is more than 1, the last set is left for test.
- sort_atomsbool
Sort atoms by atom types. Required to enable when the data is directly feeded to descriptors except mixed types.
Methods
add(key, ndof[, atomic, must, high_prec, ...])Add a data item that to be loaded.
avg(key)Return the average value of an item.
check_batch_size(batch_size)Check if the system can get a batch of data with batch_size frames.
check_test_size(test_size)Check if the system can get a test dataset with test_size frames.
Get atom types.
get_batch(batch_size)Get a batch of data with batch_size frames.
Get the data_dict.
get_item_torch(index)Get a single frame data .
Get number of atoms.
get_natoms_vec(ntypes)Get number of atoms and number of atoms in different types.
Number of atom types in the system.
get_numb_batch(batch_size, set_idx)Get the number of batches in a set.
Get number of training sets.
get_sys_numb_batch(batch_size)Get the number of batches in the data system.
get_test([ntests])Get the test data with ntests frames.
Get the type map.
reduce(key_out, key_in)Generate a new item from the reduction of another atom.
reformat_data_torch(data)Modify the data format for the requirements of Torch backend.
reset_get_batch
- add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None, output_natoms_for_type_sel: bool = False)[source]
Add a data item that to be loaded.
- Parameters
- key
The key of the item. The corresponding data is stored in sys_path/set.*/key.npy
- ndof
The number of dof
- atomic
The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof
- must
The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0
- high_prec
Load the data and store in float64, otherwise in float32
- type_sel
Select certain type of atoms
- repeat
The data will be repeated repeat times.
- default
float, default=0. default value of data
- dtype
np.dtype,optional the dtype of data, overwrites high_prec if provided
- output_natoms_for_type_selbool,
optional if True and type_sel is True, the atomic dimension will be natoms instead of nsel
- check_batch_size(batch_size)[source]
Check if the system can get a batch of data with batch_size frames.
- check_test_size(test_size)[source]
Check if the system can get a test dataset with test_size frames.
- get_batch(batch_size: int) dict[source]
Get a batch of data with batch_size frames. The frames are randomly picked from the data system.
- Parameters
- batch_size
size of the batch
- get_item_torch(index: int) dict[source]
Get a single frame data . The frame is picked from the data system by index. The index is coded across all the sets.
- Parameters
- index
index of the frame
- get_natoms_vec(ntypes: int)[source]
Get number of atoms and number of atoms in different types.
- Parameters
- ntypes
Number of types (may be larger than the actual number of types in the system).
- Returns
natomsnatoms[0]: number of local atoms natoms[1]: total number of atoms held by this processor natoms[i]: 2 <= i < Ntypes+2, number of type i atoms
- get_test(ntests: int = -1) dict[source]
Get the test data with ntests frames.
- Parameters
- ntests
Size of the test data set. If ntests is -1, all test data will be get.
- reduce(key_out: str, key_in: str)[source]
Generate a new item from the reduction of another atom.
- Parameters
- key_out
The name of the reduced item
- key_in
The name of the data item to be reduced
deepmd.tf.utils.data_system module
Alias for backward compatibility.
- class deepmd.tf.utils.data_system.DeepmdDataSystem(systems: List[str], batch_size: int, test_size: int, rcut: Optional[float] = None, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set=False, sys_probs=None, auto_prob_style='prob_sys_size', sort_atoms: bool = True)[source]
Bases:
objectClass for manipulating many data systems.
It is implemented with the help of DeepmdData
- Attributes
default_meshMesh for each system.
Methods
add(key, ndof[, atomic, must, high_prec, ...])Add a data item that to be loaded.
add_dict(adict)Add items to the data system by a dict.
get_batch([sys_idx])Get a batch of data from the data systems.
Get a batch of data from the data systems in the mixed way.
Get the batch size.
get_batch_standard([sys_idx])Get a batch of data from the data systems in the standard way.
Get the total number of batches.
Get the number of data systems.
Get the number of types.
get_sys(idx)Get a certain data system.
get_sys_ntest([sys_idx])Get number of tests for the currently selected system, or one defined by sys_idx.
get_test([sys_idx, n_test])Get test data from the the data systems.
Get the type map.
reduce(key_out, key_in)Generate a new item from the reduction of another atom.
compute_energy_shift
get_data_dict
print_summary
set_sys_probs
- add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None, output_natoms_for_type_sel: bool = False)[source]
Add a data item that to be loaded.
- Parameters
- key
The key of the item. The corresponding data is stored in sys_path/set.*/key.npy
- ndof
The number of dof
- atomic
The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof
- must
The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0
- high_prec
Load the data and store in float64, otherwise in float32
- type_sel
Select certain type of atoms
- repeat
The data will be repeated repeat times.
- default, default=0.
Default value of data
- dtype
The dtype of data, overwrites high_prec if provided
- output_natoms_for_type_selbool
If True and type_sel is True, the atomic dimension will be natoms instead of nsel
- add_dict(adict: dict) None[source]
Add items to the data system by a dict. adict should have items like .. code-block:: python.
- adict[key] = {
“ndof”: ndof, “atomic”: atomic, “must”: must, “high_prec”: high_prec, “type_sel”: type_sel, “repeat”: repeat,
}
For the explaination of the keys see add
- get_batch(sys_idx: Optional[int] = None) dict[source]
Get a batch of data from the data systems.
- Parameters
- sys_idx
int The index of system from which the batch is get. If sys_idx is not None, sys_probs and auto_prob_style are ignored If sys_idx is None, automatically determine the system according to sys_probs or auto_prob_style, see the following. This option does not work for mixed systems.
- sys_idx
- Returns
dictThe batch data
- get_batch_mixed() dict[source]
Get a batch of data from the data systems in the mixed way.
- Returns
dictThe batch data
- get_batch_standard(sys_idx: Optional[int] = None) dict[source]
Get a batch of data from the data systems in the standard way.
- get_sys(idx: int) DeepmdData[source]
Get a certain data system.
- get_sys_ntest(sys_idx=None)[source]
Get number of tests for the currently selected system, or one defined by sys_idx.
- get_test(sys_idx: Optional[int] = None, n_test: int = -1)[source]
Get test data from the the data systems.
- Parameters
- sys_idx
The test dat of system with index sys_idx will be returned. If is None, the currently selected system will be returned.
- n_test
Number of test data. If set to -1 all test data will be get.
deepmd.tf.utils.errors module
deepmd.tf.utils.finetune module
deepmd.tf.utils.graph module
- deepmd.tf.utils.graph.get_attention_layer_nodes_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the attention layer nodes with the given tf.GraphDef object.
- deepmd.tf.utils.graph.get_attention_layer_variables_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the attention layer variables with the given tf.GraphDef object.
- deepmd.tf.utils.graph.get_embedding_net_nodes(model_file: str, suffix: str = '') Dict[source]
Get the embedding net nodes with the given frozen model(model_file).
- deepmd.tf.utils.graph.get_embedding_net_nodes_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the embedding net nodes with the given tf.GraphDef object.
- deepmd.tf.utils.graph.get_embedding_net_variables(model_file: str, suffix: str = '') Dict[source]
Get the embedding net variables with the given frozen model(model_file).
- deepmd.tf.utils.graph.get_embedding_net_variables_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the embedding net variables with the given tf.GraphDef object.
- deepmd.tf.utils.graph.get_extra_embedding_net_suffix(type_one_side: bool)[source]
Get the extra embedding net suffix according to the value of type_one_side.
- Parameters
- type_one_side
The value of type_one_side
- Returns
strThe extra embedding net suffix
- deepmd.tf.utils.graph.get_extra_embedding_net_variables_from_graph_def(graph_def: GraphDef, suffix: str, extra_suffix: str, layer_size: int)[source]
Get extra embedding net variables from the given tf.GraphDef object. The “extra embedding net” means the embedding net with only type embeddings input, which occurs in “se_atten_v2” and “se_a_ebd_v2” descriptor.
- Parameters
- Returns
DictThe extra embedding net variables within the given tf.GraphDef object
- deepmd.tf.utils.graph.get_fitting_net_nodes(model_file: str) Dict[source]
Get the fitting net nodes with the given frozen model(model_file).
- Parameters
- model_file
The input frozen model path
- Returns
DictThe fitting net nodes with the given frozen model
- deepmd.tf.utils.graph.get_fitting_net_nodes_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the fitting net nodes with the given tf.GraphDef object.
- Parameters
- graph_def
The input tf.GraphDef object
- suffix
suffix of the scope
- Returns
DictThe fitting net nodes within the given tf.GraphDef object
- deepmd.tf.utils.graph.get_fitting_net_variables(model_file: str, suffix: str = '') Dict[source]
Get the fitting net variables with the given frozen model(model_file).
- Parameters
- model_file
The input frozen model path
- suffix
suffix of the scope
- Returns
DictThe fitting net variables within the given frozen model
- deepmd.tf.utils.graph.get_fitting_net_variables_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the fitting net variables with the given tf.GraphDef object.
- Parameters
- graph_def
The input tf.GraphDef object
- suffix
suffix of the scope
- Returns
DictThe fitting net variables within the given tf.GraphDef object
- deepmd.tf.utils.graph.get_pattern_nodes_from_graph_def(graph_def: GraphDef, pattern: str) Dict[source]
Get the pattern nodes with the given tf.GraphDef object.
- Parameters
- graph_def
The input tf.GraphDef object
- pattern
The node pattern within the graph_def
- Returns
DictThe fitting net nodes within the given tf.GraphDef object
- deepmd.tf.utils.graph.get_tensor_by_name(model_file: str, tensor_name: str) Tensor[source]
Load tensor value from the frozen model(model_file).
- deepmd.tf.utils.graph.get_tensor_by_name_from_graph(graph: Graph, tensor_name: str) Tensor[source]
Load tensor value from the given tf.Graph object.
- deepmd.tf.utils.graph.get_tensor_by_type(node, data_type: dtype) Tensor[source]
Get the tensor value within the given node according to the input data_type.
- Parameters
- node
The given tensorflow graph node
- data_type
The data type of the node
- Returns
tf.TensorThe tensor value of the given node
- deepmd.tf.utils.graph.get_type_embedding_net_nodes_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the type embedding net nodes with the given tf.GraphDef object.
- deepmd.tf.utils.graph.get_type_embedding_net_variables_from_graph_def(graph_def: GraphDef, suffix: str = '') Dict[source]
Get the type embedding net variables with the given tf.GraphDef object.
- deepmd.tf.utils.graph.get_variables_from_graph_def_as_numpy_array(graph_def: GraphDef, pattern: str)[source]
Get variables from the given tf.GraphDef object, with numpy array returns.
- Parameters
- graph_def
The input tf.GraphDef object
- pattern
str The name of variable
- Returns
np.ndarrayThe numpy array of the variable
deepmd.tf.utils.learning_rate module
- class deepmd.tf.utils.learning_rate.LearningRateExp(start_lr: float, stop_lr: float = 5e-08, decay_steps: int = 5000, decay_rate: float = 0.95)[source]
Bases:
objectThe exponentially decaying learning rate.
The learning rate at step \(t\) is given by
\[\alpha(t) = \alpha_0 \lambda ^ { t / \tau }\]where \(\alpha\) is the learning rate, \(\alpha_0\) is the starting learning rate, \(\lambda\) is the decay rate, and \(\tau\) is the decay steps.
- Parameters
- start_lr
Starting learning rate \(\alpha_0\)
- stop_lr
Stop learning rate \(\alpha_1\)
- decay_steps
Learning rate decay every this number of steps \(\tau\)
- decay_rate
The decay rate \(\lambda\). If stop_step is provided in build, then it will be determined automatically and overwritten.
Methods
build(global_step[, stop_step])Build the learning rate.
start_lr()Get the start lr.
value(step)Get the lr at a certain step.
- build(global_step: Tensor, stop_step: Optional[int] = None) Tensor[source]
Build the learning rate.
- Parameters
- global_step
The tf Tensor prividing the global training step
- stop_step
The stop step. If provided, the decay_rate will be determined automatically and overwritten.
- Returns
learning_rateThe learning rate
deepmd.tf.utils.multi_init module
deepmd.tf.utils.neighbor_stat module
- class deepmd.tf.utils.neighbor_stat.NeighborStat(ntypes: int, rcut: float, mixed_type: bool = False)[source]
Bases:
NeighborStatClass for getting training data information.
It loads data from DeepmdData object, and measures the data info, including neareest nbor distance between atoms, max nbor size of atoms and the output data range of the environment matrix.
- Parameters
- ntypes
The num of atom types
- rcut
The cut-off radius
- mixed_typebool,
optional, default=False Treat all types as a single type.
Methods
build()Build the graph.
get_stat(data)Get the data statistics of the training data, including nearest nbor distance between atoms, max nbor size of atoms.
iterator(data)Produce data.
- class deepmd.tf.utils.neighbor_stat.NeighborStatOP(ntypes: int, rcut: float, mixed_types: bool)[source]
Bases:
objectClass for getting neighbor statics data information.
- Parameters
- ntypes
The num of atom types
- rcut
The cut-off radius
- mixed_typesbool,
optional If True, treat neighbors of all types as a single type.
Methods
build(coord, atype, cell, pbc)Calculate the nearest neighbor distance between atoms, maximum nbor size of atoms and the output data range of the environment matrix.
deepmd.tf.utils.network module
- deepmd.tf.utils.network.embedding_net(xx, network_size, precision, activation_fn=<function tanh>, resnet_dt=False, name_suffix='', stddev=1.0, bavg=0.0, seed=None, trainable=True, uniform_seed=False, initial_variables=None, mixed_prec=None)[source]
The embedding network.
The embedding network function \(\mathcal{N}\) is constructed by is the composition of multiple layers \(\mathcal{L}^{(i)}\):
\[\mathcal{N} = \mathcal{L}^{(n)} \circ \mathcal{L}^{(n-1)} \circ \cdots \circ \mathcal{L}^{(1)}\]A layer \(\mathcal{L}\) is given by one of the following forms, depending on the number of nodes: [1]
\[\begin{split}\mathbf{y}=\mathcal{L}(\mathbf{x};\mathbf{w},\mathbf{b})= \begin{cases} \boldsymbol{\phi}(\mathbf{x}^T\mathbf{w}+\mathbf{b}) + \mathbf{x}, & N_2=N_1 \\ \boldsymbol{\phi}(\mathbf{x}^T\mathbf{w}+\mathbf{b}) + (\mathbf{x}, \mathbf{x}), & N_2 = 2N_1\\ \boldsymbol{\phi}(\mathbf{x}^T\mathbf{w}+\mathbf{b}), & \text{otherwise} \\ \end{cases}\end{split}\]where \(\mathbf{x} \in \mathbb{R}^{N_1}\) is the input vector and \(\mathbf{y} \in \mathbb{R}^{N_2}\) is the output vector. \(\mathbf{w} \in \mathbb{R}^{N_1 \times N_2}\) and \(\mathbf{b} \in \mathbb{R}^{N_2}\) are weights and biases, respectively, both of which are trainable if trainable is True. \(\boldsymbol{\phi}\) is the activation function.
- Parameters
- xx
Tensor Input tensor \(\mathbf{x}\) of shape [-1,1]
- network_size
listofint Size of the embedding network. For example [16,32,64]
- precision:
Precision of network weights. For example, tf.float64
- activation_fn:
Activation function \(\boldsymbol{\phi}\)
- resnet_dtbool
Using time-step in the ResNet construction
- name_suffix
str The name suffix append to each variable.
- stddev
float Standard deviation of initializing network parameters
- bavg
float Mean of network intial bias
- seed
int Random seed for initializing network parameters
- trainablebool
If the network is trainable
- uniform_seedbool
Only for the purpose of backward compatibility, retrieves the old behavior of using the random seed
- initial_variables
dict The input dict which stores the embedding net variables
- mixed_prec
The input dict which stores the mixed precision setting for the embedding net
- xx
References
- 1
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identitymappings in deep residual networks. InComputer Vision - ECCV 2016,pages 630-645. Springer International Publishing, 2016.
- deepmd.tf.utils.network.one_layer(inputs, outputs_size, activation_fn=<function tanh>, precision=tf.float64, stddev=1.0, bavg=0.0, name='linear', scope='', reuse=None, seed=None, use_timestep=False, trainable=True, useBN=False, uniform_seed=False, initial_variables=None, mixed_prec=None, final_layer=False)[source]
- deepmd.tf.utils.network.variable_summaries(var: VariableV1, name: str)[source]
Attach a lot of summaries to a Tensor (for TensorBoard visualization).
- Parameters
- var
tf.Variable [description]
- name
str variable name
- var
deepmd.tf.utils.nlist module
- deepmd.tf.utils.nlist.extend_coord_with_ghosts(coord: Tensor, atype: Tensor, cell: Tensor, rcut: float, pbc: Tensor)[source]
Extend the coordinates of the atoms by appending peridoc images. The number of images is large enough to ensure all the neighbors within rcut are appended.
- Parameters
- Returns
deepmd.tf.utils.pair_tab module
Alias for backward compatibility.
- class deepmd.tf.utils.pair_tab.PairTab(filename: str, rcut: Optional[float] = None)[source]
Bases:
objectPairwise tabulated potential.
- Parameters
- filename
File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.
Methods
get()Get the serialized table.
reinit(filename[, rcut])Initialize the tabulated interaction.
deserialize
serialize
- reinit(filename: str, rcut: Optional[float] = None) None[source]
Initialize the tabulated interaction.
- Parameters
- filename
File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.
deepmd.tf.utils.parallel_op module
- class deepmd.tf.utils.parallel_op.ParallelOp(builder: Callable[[...], Tuple[Dict[str, Tensor], Tuple[Tensor]]], nthreads: Optional[int] = None, config: Optional[ConfigProto] = None)[source]
Bases:
objectRun an op with data parallelism.
- Parameters
Examples
>>> from deepmd.tf.env import tf >>> from deepmd.tf.utils.parallel_op import ParallelOp >>> def builder(): ... x = tf.placeholder(tf.int32, [1]) ... return {"x": x}, (x + 1) >>> p = ParallelOp(builder, nthreads=4) >>> def feed(): ... for ii in range(10): ... yield {"x": [ii]} >>> print(*p.generate(tf.Session(), feed())) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
Methods
generate(sess, feed)Returns a generator.
deepmd.tf.utils.path module
Alias for backward compatibility.
- class deepmd.tf.utils.path.DPH5Path(path: str, mode: str = 'r')[source]
Bases:
DPPathThe path class to data system (DeepmdData) for HDF5 files.
Notes
- OS - HDF5 relationship:
directory - Group file - Dataset
- Attributes
nameName of the path.
Methods
glob(pattern)Search path using the glob pattern.
is_dir()Check if self is directory.
is_file()Check if self is file.
Load NumPy array.
load_txt([dtype])Load NumPy array from text.
mkdir([parents, exist_ok])Make directory.
rglob(pattern)This is like calling
DPPath.glob()with **/ added in front of the given relative pattern.save_numpy(arr)Save NumPy array.
- load_numpy() ndarray[source]
Load NumPy array.
- Returns
np.ndarrayloaded NumPy array
- load_txt(dtype: Optional[dtype] = None, **kwargs) ndarray[source]
Load NumPy array from text.
- Returns
np.ndarrayloaded NumPy array
- rglob(pattern: str) List[DPPath][source]
This is like calling
DPPath.glob()with **/ added in front of the given relative pattern.
- save_numpy(arr: ndarray) None[source]
Save NumPy array.
- Parameters
- arr
np.ndarray NumPy array
- arr
- class deepmd.tf.utils.path.DPOSPath(path: str, mode: str = 'r')[source]
Bases:
DPPathThe OS path class to data system (DeepmdData) for real directories.
Methods
glob(pattern)Search path using the glob pattern.
is_dir()Check if self is directory.
is_file()Check if self is file.
Load NumPy array.
load_txt(**kwargs)Load NumPy array from text.
mkdir([parents, exist_ok])Make directory.
rglob(pattern)This is like calling
DPPath.glob()with **/ added in front of the given relative pattern.save_numpy(arr)Save NumPy array.
- load_numpy() ndarray[source]
Load NumPy array.
- Returns
np.ndarrayloaded NumPy array
- load_txt(**kwargs) ndarray[source]
Load NumPy array from text.
- Returns
np.ndarrayloaded NumPy array
- rglob(pattern: str) List[DPPath][source]
This is like calling
DPPath.glob()with **/ added in front of the given relative pattern.
- save_numpy(arr: ndarray) None[source]
Save NumPy array.
- Parameters
- arr
np.ndarray NumPy array
- arr
- class deepmd.tf.utils.path.DPPath(path: str, mode: str = 'r')[source]
Bases:
ABCThe path class to data system (DeepmdData).
Methods
glob(pattern)Search path using the glob pattern.
is_dir()Check if self is directory.
is_file()Check if self is file.
Load NumPy array.
load_txt(**kwargs)Load NumPy array from text.
mkdir([parents, exist_ok])Make directory.
rglob(pattern)This is like calling
DPPath.glob()with **/ added in front of the given relative pattern.save_numpy(arr)Save NumPy array.
- abstract load_numpy() ndarray[source]
Load NumPy array.
- Returns
np.ndarrayloaded NumPy array
- abstract load_txt(**kwargs) ndarray[source]
Load NumPy array from text.
- Returns
np.ndarrayloaded NumPy array
- abstract rglob(pattern: str) List[DPPath][source]
This is like calling
DPPath.glob()with **/ added in front of the given relative pattern.
- abstract save_numpy(arr: ndarray) None[source]
Save NumPy array.
- Parameters
- arr
np.ndarray NumPy array
- arr
deepmd.tf.utils.plugin module
Alias for backward compatibility.
- class deepmd.tf.utils.plugin.Plugin[source]
Bases:
objectA class to register and restore plugins.
Examples
>>> plugin = Plugin() >>> @plugin.register("xx") def xxx(): pass >>> print(plugin.plugins["xx"])
Methods
get_plugin(key)Visit a plugin by key.
register(key)Register a plugin.
- class deepmd.tf.utils.plugin.PluginVariant(*args, **kwargs)[source]
Bases:
objectA class to remove type from input arguments.
- class deepmd.tf.utils.plugin.VariantABCMeta(name, bases, namespace, **kwargs)[source]
Bases:
VariantMeta,ABCMetaMethods
__call__(*args, **kwargs)Remove type and keys that starts with underline.
mro(/)Return a type's method resolution order.
register(subclass)Register a virtual subclass of an ABC.
deepmd.tf.utils.random module
Alias for backward compatibility.
- deepmd.tf.utils.random.choice(a: Union[ndarray, int], size: Optional[Union[int, Tuple[int, ...]]] = None, replace: bool = True, p: Optional[ndarray] = None)[source]
Generates a random sample from a given 1-D array.
- Parameters
- a1-D array_like or
int If an ndarray, a random sample is generated from its elements. If an int, the random sample is generated as if it were np.arange(a)
- size
intortupleofints,optional Output shape. If the given shape is, e.g., (m, n, k), then m * n * k samples are drawn. Default is None, in which case a single value is returned.
- replacebool,
optional Whether the sample is with or without replacement. Default is True, meaning that a value of a can be selected multiple times.
- p1-D array_like,
optional The probabilities associated with each entry in a. If not given, the sample assumes a uniform distribution over all entries in a.
- a1-D array_like or
- Returns
np.ndarrayarrays with results and their shapes
- deepmd.tf.utils.random.random(size=None)[source]
Return random floats in the half-open interval [0.0, 1.0).
- Parameters
- size
Output shape.
- Returns
np.ndarrayArrays with results and their shapes.
- deepmd.tf.utils.random.seed(val: Optional[int] = None)[source]
Seed the generator.
- Parameters
- val
int Seed.
- val
- deepmd.tf.utils.random.shuffle(x: ndarray)[source]
Modify a sequence in-place by shuffling its contents.
- Parameters
- x
np.ndarray The array or list to be shuffled.
- x
deepmd.tf.utils.region module
deepmd.tf.utils.serialization module
deepmd.tf.utils.sess module
deepmd.tf.utils.spin module
- class deepmd.tf.utils.spin.Spin(use_spin: Optional[List[bool]] = None, spin_norm: Optional[List[float]] = None, virtual_len: Optional[List[float]] = None)[source]
Bases:
objectClass for spin.
- Parameters
- use_spin
Whether to use atomic spin model for each atom type
- spin_norm
The magnitude of atomic spin for each atom type with spin
- virtual_len
The distance between virtual atom representing spin and its corresponding real atom for each atom type with spin
Methods
build([reuse, suffix])Build the computational graph for the spin.
Returns the number of atom types which contain spin.
Returns the list of magnitude of atomic spin for each atom type.
Returns the list of whether to use spin for each atom type.
Returns the list of distance between real atom and virtual atom for each atom type.
- build(reuse=None, suffix='')[source]
Build the computational graph for the spin.
- Parameters
- reuse
The weights in the networks should be reused when get the variable.
- suffix
Name suffix to identify this descriptor
- Returns
embedded_typesThe computational graph for embedded types
deepmd.tf.utils.tabulate module
- class deepmd.tf.utils.tabulate.DPTabulate(descrpt: ~deepmd.tf.descriptor.descriptor.Descriptor, neuron: ~typing.List[int], graph: ~tensorflow.python.framework.ops.Graph, graph_def: ~tensorflow.core.framework.graph_pb2.GraphDef, type_one_side: bool = False, exclude_types: ~typing.List[~typing.List[int]] = [], activation_fn: ~typing.Callable[[~tensorflow.python.framework.tensor.Tensor], ~tensorflow.python.framework.tensor.Tensor] = <function tanh>, suffix: str = '')[source]
Bases:
objectClass for tabulation.
Compress a model, which including tabulating the embedding-net. The table is composed of fifth-order polynomial coefficients and is assembled from two sub-tables. The first table takes the stride(parameter) as it’s uniform stride, while the second table takes 10 * stride as it’s uniform stride The range of the first table is automatically detected by deepmd-kit, while the second table ranges from the first table’s upper boundary(upper) to the extrapolate(parameter) * upper.
- Parameters
- descrpt
Descriptor of the original model
- neuron
Number of neurons in each hidden layers of the embedding net \(\\mathcal{N}\)
- graph
tf.Graph The graph of the original model
- graph_def
tf.GraphDef The graph_def of the original model
- type_one_side
Try to build N_types tables. Otherwise, building N_types^2 tables
- exclude_types
List[List[int]] The excluded pairs of types which have no interaction with each other. For example, [[0, 1]] means no interaction between type 0 and type 1.
- activation_function
The activation function in the embedding net. Supported options are {“tanh”,”gelu”} in common.ACTIVATION_FN_DICT.
- suffix
str,optional The suffix of the scope
Methods
build(min_nbor_dist, extrapolate, stride0, ...)Build the tables for model compression.
- build(min_nbor_dist: float, extrapolate: float, stride0: float, stride1: float) Tuple[Dict[str, int], Dict[str, int]][source]
Build the tables for model compression.
- Parameters
- min_nbor_dist
The nearest distance between neighbor atoms
- extrapolate
The scale of model extrapolation
- stride0
The uniform stride of the first table
- stride1
The uniform stride of the second table
- Returns
deepmd.tf.utils.type_embed module
- class deepmd.tf.utils.type_embed.TypeEmbedNet(neuron: List[int] = [], resnet_dt: bool = False, activation_function: Optional[str] = 'tanh', precision: str = 'default', trainable: bool = True, seed: Optional[int] = None, uniform_seed: bool = False, padding: bool = False, **kwargs)[source]
Bases:
objectType embedding network.
- Parameters
- neuron
list[int] Number of neurons in each hidden layers of the embedding net
- resnet_dt
Time-step dt in the resnet construction: y = x + dt * phi (Wx + b)
- activation_function
The activation function in the embedding net. Supported options are “relu”, “relu6”, “softplus”, “sigmoid”, “tanh”, “gelu”, “gelu_tf”, “None”, “none”.
- precision
The precision of the embedding net parameters. Supported options are “default”, “float16”, “float32”, “float64”, “bfloat16”.
- trainable
If the weights of embedding net are trainable.
- seed
Random seed for initializing the network parameters.
- uniform_seed
Only for the purpose of backward compatibility, retrieves the old behavior of using the random seed
- padding
Concat the zero padding to the output, as the default embedding of empty type.
- neuron
Methods
build(ntypes[, reuse, suffix])Build the computational graph for the descriptor.
init_variables(graph, graph_def[, suffix, ...])Init the type embedding net variables with the given dict.
- build(ntypes: int, reuse=None, suffix='')[source]
Build the computational graph for the descriptor.
- Parameters
- ntypes
Number of atom types.
- reuse
The weights in the networks should be reused when get the variable.
- suffix
Name suffix to identify this descriptor
- Returns
embedded_typesThe computational graph for embedded types
- init_variables(graph: Graph, graph_def: GraphDef, suffix='', model_type='original_model') None[source]
Init the type embedding net variables with the given dict.
- Parameters
- graph
tf.Graph The input frozen model graph
- graph_def
tf.GraphDef The input frozen model graph_def
- suffix
Name suffix to identify this descriptor
- model_type
Indicator of whether this model is a compressed model
- graph
- deepmd.tf.utils.type_embed.embed_atom_type(ntypes: int, natoms: Tensor, type_embedding: Tensor)[source]
Make the embedded type for the atoms in system. The atoms are assumed to be sorted according to the type, thus their types are described by a tf.Tensor natoms, see explanation below.
- Parameters
- ntypes:
Number of types.
- natoms:
The number of atoms. This tensor has the length of Ntypes + 2 natoms[0]: number of local atoms natoms[1]: total number of atoms held by this processor natoms[i]: 2 <= i < Ntypes+2, number of type i atoms
- type_embedding:
The type embedding. It has the shape of [ntypes, embedding_dim]
- Returns
atom_embeddingThe embedded type of each atom. It has the shape of [numb_atoms, embedding_dim]
deepmd.tf.utils.update_sel module
- class deepmd.tf.utils.update_sel.UpdateSel[source]
Bases:
BaseUpdateSel- Attributes
- neighbor_stat
Methods
get_min_nbor_dist
get_nbor_stat
get_rcut
get_sel
get_type_map
hook
parse_auto_sel
parse_auto_sel_ratio
update_one_sel
wrap_up_4
- property neighbor_stat: Type[NeighborStat]
deepmd.tf.utils.weight_avg module
Alias for backward compatibility.