Creating and manipulating datatypes which describe elements of a dataset (H5T)
General Datatype Operations
Atomic Datatype Properties
Compound Datatype Properties
Variable-length Array Datatypes
* Use of these functions is deprecated in Release 1.8.0.
The Datatype interface, H5T, provides a mechanism to describe the storage format of individual data points of a data set and is hopefully designed in such a way as to allow new features to be easily added without disrupting applications that use the data type interface. A dataset (the H5D interface) is composed of a collection or raw data points of homogeneous type organized according to the data space (the H5S interface).
A datatype is a collection of datatype properties, all of which can be stored on disk, and which when taken as a whole, provide complete information for data conversion to or from that datatype. The interface provides functions to set and query properties of a datatype.
A data point is an instance of a datatype, which is an instance of a type class. We have defined a set of type classes and properties which can be extended at a later time. The atomic type classes are those which describe types which cannot be decomposed at the datatype interface level; all other classes are compound.
See HDF5 Datatypes in the HDF5 User’s Guide for more information.
List of pre-defined datatypes in HDF5
Creating variable-length string datatypes
As the term implies, variable-length strings are strings of varying lengths; they can be arbitrarily long, anywhere from 1 character to thousands of characters.
HDF5 provides the ability to create a variable-length string datatype. Like all string datatypes, this type is based on the atomic string datatype:
H5T_C_S1 in C or
H5T_FORTRAN_S1 in Fortran. While these datatypes default to one character in size, they can be resized to specific fixed lengths or to variable length.
Variable-length strings will transparently accommodate ASCII strings or UTF-8 strings. This characteristic is set with H5T_SET_CSET in the process of creating the datatype.
The following HDF5 calls create a C-style variable-length string datatype,
vls_type_c_id = H5Tcopy(H5T_C_S1) status = H5Tset_size(vls_type_c_id, H5T_VARIABLE)
In a C environment, variable-length strings will always be NULL-terminated, so the buffer to hold such a string must be one byte larger than the string itself to accommodate the NULL terminator.
In Fortran, strings are normally of fixed length. Variable-length strings come into play only when data is shared with a C application that uses them. For such situations, the datatype class
H5T_STRING is predefined by the HDF5 library to accommodate variable-length strings. The first HDF5 call below creates a Fortran string,
vls_type_f_id, that will handle variable-length string data. The second call sets the string padding value to space padding:
h5tcopy_f(H5T_STRING, vls_type_f_id, hdferr) h5tset_strpad_f(vls_type_f_id, H5T_STR_SPACEPAD_F, hdferr)
While Fortran-style strings are generally space-padded, they may be NULL-terminated in cases where the data is also used in a C environment.
Note: Under the covers, variable-length strings are stored in a heap, potentially impacting efficiency in the following ways:
- Heap storage requires more space than regular raw data storage.
- Heap access generally reduces I/O efficiency because it requires individual read or write operations for each data element rather than one read or write per dataset or per data selection.
- Chunking and filters, including compression, are not available for heaps.