deepmd.tf.utils.network

Module Contents

Functions

one_layer_rand_seed_shift()
one_layer(inputs, outputs_size[, activation_fn, ...])
layer_norm_tf(x, shape[, weight, bias, eps])	Layer normalization implementation in TensorFlow.
layernorm(inputs, outputs_size[, precision, name, ...])
embedding_net_rand_seed_shift(network_size)
embedding_net(xx, network_size, precision[, ...])	The embedding network.
variable_summaries(var, name)	Attach a lot of summaries to a Tensor (for TensorBoard visualization).

deepmd.tf.utils.network.one_layer_rand_seed_shift()

deepmd.tf.utils.network.one_layer(inputs, outputs_size, activation_fn=tf.nn.tanh, precision=GLOBAL_TF_FLOAT_PRECISION, stddev=1.0, bavg=0.0, name='linear', scope='', reuse=None, seed=None, use_timestep=False, trainable=True, useBN=False, uniform_seed=False, initial_variables=None, mixed_prec=None, final_layer=False)

deepmd.tf.utils.network.layer_norm_tf(x, shape, weight=None, bias=None, eps=1e-05)

Layer normalization implementation in TensorFlow.

Parameters:

xtf.Tensor

The input tensor.

shapetuple

The shape of the weight and bias tensors.

weighttf.Tensor

The weight tensor.

biastf.Tensor

The bias tensor.

epsfloat

A small value added to prevent division by zero.

Returns:

tf.Tensor

The normalized output tensor.

deepmd.tf.utils.network.layernorm(inputs, outputs_size, precision=GLOBAL_TF_FLOAT_PRECISION, name='linear', scope='', reuse=None, seed=None, uniform_seed=False, uni_init=True, eps=1e-05, trainable=True, initial_variables=None)

deepmd.tf.utils.network.embedding_net_rand_seed_shift(network_size)

deepmd.tf.utils.network.embedding_net(xx, network_size, precision, activation_fn=tf.nn.tanh, resnet_dt=False, name_suffix='', stddev=1.0, bavg=0.0, seed=None, trainable=True, uniform_seed=False, initial_variables=None, mixed_prec=None)

The embedding network.

The embedding network function \(\mathcal{N}\) is constructed by is the composition of multiple layers \(\mathcal{L}^{(i)}\):

\[\mathcal{N} = \mathcal{L}^{(n)} \circ \mathcal{L}^{(n-1)} \circ \cdots \circ \mathcal{L}^{(1)}\]

A layer \(\mathcal{L}\) is given by one of the following forms, depending on the number of nodes: [1]

\[\begin{split}\mathbf{y}=\mathcal{L}(\mathbf{x};\mathbf{w},\mathbf{b})= \begin{cases} \boldsymbol{\phi}(\mathbf{x}^T\mathbf{w}+\mathbf{b}) + \mathbf{x}, & N_2=N_1 \\ \boldsymbol{\phi}(\mathbf{x}^T\mathbf{w}+\mathbf{b}) + (\mathbf{x}, \mathbf{x}), & N_2 = 2N_1\\ \boldsymbol{\phi}(\mathbf{x}^T\mathbf{w}+\mathbf{b}), & \text{otherwise} \\ \end{cases}\end{split}\]

where \(\mathbf{x} \in \mathbb{R}^{N_1}\) is the input vector and \(\mathbf{y} \in \mathbb{R}^{N_2}\) is the output vector. \(\mathbf{w} \in \mathbb{R}^{N_1 \times N_2}\) and \(\mathbf{b} \in \mathbb{R}^{N_2}\) are weights and biases, respectively, both of which are trainable if trainable is True. \(\boldsymbol{\phi}\) is the activation function.

Parameters:

xxTensor

Input tensor \(\mathbf{x}\) of shape [-1,1]

network_sizelist of int

Size of the embedding network. For example [16,32,64]

precision:

Precision of network weights. For example, tf.float64

activation_fn:

Activation function \(\boldsymbol{\phi}\)

resnet_dtbool

Using time-step in the ResNet construction

name_suffixstr

The name suffix append to each variable.

stddevfloat

Standard deviation of initializing network parameters

bavgfloat

Mean of network intial bias

seedint

Random seed for initializing network parameters

trainablebool

If the network is trainable

uniform_seedbool

Only for the purpose of backward compatibility, retrieves the old behavior of using the random seed

initial_variablesdict

The input dict which stores the embedding net variables

mixed_prec

The input dict which stores the mixed precision setting for the embedding net

References

[1]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identitymappings in deep residual networks. InComputer Vision - ECCV 2016,pages 630-645. Springer International Publishing, 2016.

deepmd.tf.utils.network.variable_summaries(var: deepmd.tf.env.tf.Variable, name: str)

Attach a lot of summaries to a Tensor (for TensorBoard visualization).

Parameters:

vartf.Variable

[description]

namestr

variable name