Merge pull request #2461 from firedrakeproject/ksagiyam/plex_io_freez…

…e_distribution

Save/Load mesh distributions and permutations

docs/source/checkpointing.rst

@@ -22,6 +22,29 @@ to pass through a single process.
 It also supports flexible checkpointing, where one can save meshes and
 :class:`~.Function` s on :math:`N` processes and later load them on
 :math:`P` processes.
+If :math:`P == N`, the parallel distribution and entity permutation
+(reordering) of the saved mesh is recovered on the loaded mesh by default.
+The distribution and permutation data are by default stored under names
+automatically generated by Firedrake.
+
+.. warning::
+
+    If the mesh has a non-standard distribution, e.g., generated by a
+    partitioner with some non-standard parameters, it is recommended that
+    the user set the distribution name explicitly when constructing a mesh;
+    see, e.g., :py:func:`~.Mesh`.
+
+.. warning::
+
+   If :math:`P != N` or `P == N` but `distribution_parameters` dict and/or
+   `reorder` parameter are passed when loading a mesh, the saved mesh and
+   the loaded mesh (and thus the saved function and the loaded function)
+   will in general be represented by different global numbering systems and
+   they are merely guaranteed to be analytically the same; as a consequence,
+   it is currently not allowed to save the once loaded mesh or function back
+   to the same file under the same name as this would cause conflict with
+   other data stored using incompatible global numbering system.
+   We plan to remove this restriction in the future.
 
 Saving
 ------
@@ -72,6 +95,14 @@ the HDF5 installation; "h5dump -n example.h5", for instance, shows:
      group      /topologies/firedrake_mixed_meshes/meshA/firedrake_mixed_function_spaces/firedrake_function_space_meshA_CG2(None,None)_meshA_CG1(None,None)/firedrake_functions/g/0
      group      /topologies/firedrake_mixed_meshes/meshA/firedrake_mixed_function_spaces/firedrake_function_space_meshA_CG2(None,None)_meshA_CG1(None,None)/firedrake_functions/g/1
      group      /topologies/meshA_topology
+     group      /topologies/meshA_topology/distributions
+     group      /topologies/meshA_topology/distributions/firedrake_default_1_True_None_(FACET,1)
+     dataset    /topologies/meshA_topology/distributions/firedrake_default_1_True_None_(FACET,1)/chart_sizes
+     dataset    /topologies/meshA_topology/distributions/firedrake_default_1_True_None_(FACET,1)/global_point_numbers
+     dataset    /topologies/meshA_topology/distributions/firedrake_default_1_True_None_(FACET,1)/owners
+     group      /topologies/meshA_topology/distributions/firedrake_default_1_True_None_(FACET,1)/permutations
+     group      /topologies/meshA_topology/distributions/firedrake_default_1_True_None_(FACET,1)/permutations/firedrake_default_True
+     dataset    /topologies/meshA_topology/distributions/firedrake_default_1_True_None_(FACET,1)/permutations/firedrake_default_True/permutation
      group      /topologies/meshA_topology/dms
      group      /topologies/meshA_topology/dms/coordinateDM
      dataset    /topologies/meshA_topology/dms/coordinateDM/order