deepmd/lmdb format

deepmd/lmdb format#

Aliases#

deepmd/lmdb, lmdb

Implementation: LMDBFormat

Overview#

DeePMD-kit compatible LMDB format.

A single flat LMDB stores all frames from one or many systems. The same on-disk format is produced regardless of whether the source is a standard or a mixed-type system, so the output is always readable by DeePMD-kit’s LmdbDataReader.

The mixed_type keyword controls only how frames are mapped back to dpdata objects on read (see from_multi_systems()).

Examples

Write a single labeled system:

>>> import dpdata
>>> ls = dpdata.LabeledSystem("OUTCAR", fmt="vasp/outcar")
>>> ls.to("deepmd/lmdb", "data.lmdb")

Write many systems into one LMDB, forcing a global type map:

>>> ms = dpdata.MultiSystems(s1, s2, s3)
>>> ms.to("deepmd/lmdb", "data.lmdb", type_map=["H", "C", "N", "O"])

Read back as standard (per-composition) systems:

>>> ms = dpdata.MultiSystems.from_file("data.lmdb", fmt="deepmd/lmdb")

Read back keeping the full global type map on every system:

>>> ms = dpdata.MultiSystems.from_file(
...     "data.lmdb", fmt="deepmd/lmdb", mixed_type=True
... )

Note that loading through dpdata.MultiSystems normalises the atom_names order (the element set is kept, but reordered); a direct single-system load preserves the stored order.

Quick examples#

The examples use the preferred alias deepmd/lmdb; any alias listed above is equivalent.

import dpdata

# Geometry-only data
system = dpdata.System("input_file", fmt="deepmd/lmdb")

# Data with energies and forces
labeled_system = dpdata.LabeledSystem("input_file", fmt="deepmd/lmdb")

# Multiple compositions or calculation directories
systems = dpdata.MultiSystems.from_file("input_directory", fmt="deepmd/lmdb")

# Write geometry-only data
system.to("deepmd/lmdb", "output_path")

# Write labeled data
labeled_system.to("deepmd/lmdb", "output_path")

# Write multiple systems
systems.to("deepmd/lmdb", "output_path")

Conversions#

Convert from this format to System#

dpdata.System(file_name, fmt: Literal['deepmd/lmdb'] = None, **kwargs) → dpdata.system.System
dpdata.System(file_name, fmt: Literal['lmdb'] = None, **kwargs) → dpdata.system.System
dpdata.System.from_deepmd_lmdb(file_name, **kwargs) → dpdata.system.System
dpdata.System.from_lmdb(file_name, **kwargs) → dpdata.system.System

Load the first composition group as a System.

Parameters:
file_namestr, os.PathLike, or dict

LMDB directory, or an already decoded system-data dictionary used internally by MultiSystems loading.

**kwargsdict

Reader options accepted by from_multi_systems(), including mixed_type, type_map, and max_frames.

Returns:
System

converted system

Warns:
UserWarning

If the database contains more than one composition. Use dpdata.MultiSystems.from_file() to load all groups.

Convert from System to this format#

dpdata.System.to(fmt: Literal['deepmd/lmdb'], file_name, **kwargs)
dpdata.System.to(fmt: Literal['lmdb'], file_name, **kwargs)
dpdata.System.to_deepmd_lmdb(file_name, **kwargs)
dpdata.System.to_lmdb(file_name, **kwargs)

Save a single unlabeled System to an LMDB database.

Parameters:
file_namestr or os.PathLike

Destination LMDB directory.

**kwargsdict

Writer options accepted by to_multi_systems(), including map_size, frame_idx_fmt, type_map, write_batch_size, and overwrite.

Convert from LabeledSystem to this format#

dpdata.LabeledSystem.to(fmt: Literal['deepmd/lmdb'], file_name, **kwargs)
dpdata.LabeledSystem.to(fmt: Literal['lmdb'], file_name, **kwargs)
dpdata.LabeledSystem.to_deepmd_lmdb(file_name, **kwargs)
dpdata.LabeledSystem.to_lmdb(file_name, **kwargs)

Save a single LabeledSystem to an LMDB database.

Parameters:
file_namestr or os.PathLike

Destination LMDB directory.

**kwargsdict

Writer options accepted by to_multi_systems(), including map_size, frame_idx_fmt, type_map, write_batch_size, and overwrite.

Convert from this format to LabeledSystem#

dpdata.LabeledSystem(file_name, fmt: Literal['deepmd/lmdb'] = None, **kwargs) → dpdata.system.LabeledSystem
dpdata.LabeledSystem(file_name, fmt: Literal['lmdb'] = None, **kwargs) → dpdata.system.LabeledSystem
dpdata.LabeledSystem.from_deepmd_lmdb(file_name, **kwargs) → dpdata.system.LabeledSystem
dpdata.LabeledSystem.from_lmdb(file_name, **kwargs) → dpdata.system.LabeledSystem

Load the first composition group as a LabeledSystem.

Parameters:
file_namestr, os.PathLike, or dict

LMDB directory, or an already decoded system-data dictionary used internally by MultiSystems loading.

**kwargsdict

Reader options accepted by from_multi_systems(), including mixed_type, type_map, and max_frames.

Returns:
LabeledSystem

converted system

Raises:
LMDBFrameError

If the selected data has no energy labels.

Convert from this format to MultiSystems#

dpdata.MultiSystems.from_deepmd_lmdb(directory, mixed_type: 'bool' = False, type_map: 'list[str] | None' = None, max_frames: 'int | None' = 100000, **kwargs) → dpdata.system.MultiSystems
dpdata.MultiSystems.from_lmdb(directory, mixed_type: 'bool' = False, type_map: 'list[str] | None' = None, max_frames: 'int | None' = 100000, **kwargs) → dpdata.system.MultiSystems

Load systems from a flat LMDB.

Frames are grouped by atom-count composition. Atom order is canonicalized by a stable sort on the global atom type, and every registered atomic field follows the same permutation. Each composition becomes one dpdata system.

Parameters:
directorystr

Path to the LMDB directory.

mixed_typebool, optional

If False (default) each system’s atom_names is the compact set of elements it actually contains. If True every system keeps the full global type_map as atom_names (with zero counts for absent elements).

type_maplist[str], optional

Requested element table for the returned systems. When the file stores a type_map, the stored global indices are remapped to this table by element name (consistent with DeePMD-kit); every element in the file must be present in type_map. When the file has no type_map, the indices are named positionally from this argument. Defaults to the type_map stored in the file.

max_framesint or None, optional

Maximum number of frames loaded into memory. The default is 100,000. Set to None only when sufficient memory is available.

**kwargsdict

other parameters

Returns:
MultiSystems

converted system

Convert from MultiSystems to this format#

dpdata.MultiSystems.to(fmt: Literal['deepmd/lmdb'], directory, map_size: 'int' = 1099511627776, frame_idx_fmt: 'str' = '012d', type_map: 'list[str] | None' = None, write_batch_size: 'int' = 1000, overwrite: 'bool' = False, **kwargs) → dpdata.system.MultiSystems
dpdata.MultiSystems.to(fmt: Literal['lmdb'], directory, map_size: 'int' = 1099511627776, frame_idx_fmt: 'str' = '012d', type_map: 'list[str] | None' = None, write_batch_size: 'int' = 1000, overwrite: 'bool' = False, **kwargs) → dpdata.system.MultiSystems
dpdata.MultiSystems.to_deepmd_lmdb(directory, map_size: 'int' = 1099511627776, frame_idx_fmt: 'str' = '012d', type_map: 'list[str] | None' = None, write_batch_size: 'int' = 1000, overwrite: 'bool' = False, **kwargs) → dpdata.system.MultiSystems
dpdata.MultiSystems.to_lmdb(directory, map_size: 'int' = 1099511627776, frame_idx_fmt: 'str' = '012d', type_map: 'list[str] | None' = None, write_batch_size: 'int' = 1000, overwrite: 'bool' = False, **kwargs) → dpdata.system.MultiSystems

Write multiple dpdata systems to one LMDB.

Parameters:
directorystr

Output LMDB directory.

map_sizeint, optional

Maximum LMDB size in bytes. Default is 1 TiB (sparse).

frame_idx_fmtstr, optional

Format used for the per-frame integer key. Default "012d".

type_maplist[str], optional

Global element table. If None, the element list of the first system written is used (for a MultiSystems this is the union of all systems’ elements).

write_batch_sizeint, optional

Number of frames committed per LMDB write transaction.

overwritebool, optional

Whether to replace an existing destination after the new database has been written and validated. This option is supported on POSIX systems only. The default is False.

**kwargsdict

other parameters

Returns:
MultiSystems

this system