In situations where backward-compatibility is required, specifically, when libraries prior to HDF5 Release 1.8.0 may be used to read the file, groups must be created and maintained in the original style. This is HDF5’s default behavior. If backward compatibility with pre-1.8.0 libraries is not a concern, greater efficiencies can be obtained with the new-format compact and indexed groups. See Group implementations in HDF5 in the H5G API introduction (at the bottom).
H5P_SET_LOCAL_HEAP_SIZE_HINT is useful for tuning file size when files contain original-style groups with either zero members or very large numbers of members.
The original style of HDF5 groups, the only style available prior to HDF5 Release 1.8.0, was well-suited for moderate-sized groups but was not optimized for either very small or very large groups. This original style remains the default, but two new group implementations were introduced in HDF5 Release 1.8.0: compact groups to accommodate zero to small numbers of members and indexed groups for thousands or tens of thousands of members ... or millions, if that's what your application requires.
The local heap size hint,
size_hint, is a performance tuning parameter for original-style groups. As indicated above, an HDF5 group may have zero, a handful, or tens of thousands of members. Since the original style of HDF5 groups stores the metadata for all of these group members in a uniform format in a local heap, the size of that metadata (and hence, the size of the local heap) can vary wildly from group to group. To intelligently allocate space and to avoid unnecessary fragmentation of the local heap, it can be valuable to provide the library with a hint as to the local heap’s likely eventual size. This can be particularly valuable when it is known that a group will eventually have a great many members. It can also be useful in conserving space in a file when it is known that certain groups will never have any members.