Performance: Parallel I/O with Chunking Storage
HDF5 has to map the coordinate of data from the file space to the memory space in chunked storage. If the shape of the memory and file space is the same, HDF5 can optimize the mapping process significantly. Otherwise, a general mapping routine will map the coordinate one by one. We recommend that applications use the same shape for both the memory space and file space.
For example, the following case may cause bad performance:
| file space: | 2-D: the first dimension 1, the second dimension FSIZE |
| memory space: | 1-D: The fist dimension DMAX |
If you change it as follows:
| file space: | 2-D: the first dimension 1, the second dimension FSIZE |
| memory space: | 2-D: the first dimension 1, the second dimension DMAX |
Then, the performance may be much improved.