Merge pull request #146 from awslabs/hughcars/amr-doc-fixes

Assorted documentation fixes

CHANGELOG.md

@@ -13,6 +13,21 @@ The format of this changelog is based on
 
 ## In progress
 
+  - Added support for operator partial assembly for high-order finite element spaces based
+    on libCEED for mixed and non-tensor product element meshes. This option is disabled by
+    default, but can be activated using `config["Solver"]["PartialAssemblyOrder"]` set to
+    some number less than or equal to `"Order"`.
+  - Added flux-based error estimation, reported in `error-estimate.csv`. This computes the
+    difference between the numerical gradient (electrostatics) or curl (otherwise) of the
+    solution, and a smoother approximation obtained through a global mass matrix inversion.
+    The results are reported in `error-estimates.csv` within the `"Output"` folder.
+  - Added support for non axis aligned lumped ports and current sources. Key words `"X"`,
+    `"Y"`, `"Z"` and `"R"`, with optional prefix `"+"` or `"-"` still work, but now
+    directions can be specified as vectors with 3 components. Users will be warned, and
+    ultimately errored, if the specified directions do not agree with axis directions
+    discovered from the geometry.
+  - Added improved `Timer` and `BlockTimer` classes with more timing categories for
+    reporting simulation runtime.
   - Changed implementation of complex-valued linear algebra to use new `ComplexVector` and
     `ComplexOperator` types, which are based on the underlying `mfem::Vector` and
     `mfem::Operator` classes, instead of PETSc. PETSc is now fully optional and only
@@ -24,45 +39,14 @@ The format of this changelog is based on
   - Changed implementation of numeric wave ports to use MFEM's `SubMesh` functionality. As
     of [#3379](https://github.com/mfem/mfem/pull/3379) in MFEM, this has full ND and RT
     basis support. For now, support for nonconforming mesh boundaries is limited.
-  - Added support for operator partial assembly for high-order finite element spaces based
-    on libCEED for non-tensor product element meshes. This option is disabled by default,
-    but can be activated using `config["Solver"]["PartialAssemblyOrder"]` set to some number
-    less than or equal to `"Order"`.
-  - Added `config["Solver"]["Device"]` and `config["Solver"]["Backend"]` options for runtime
-    configuration of the MFEM device (CPU or GPU) and corresponding libCEED backend, with
-    suitable defaults for users.
-  - Added support for non axis aligned lumped ports and current sources. Key words `"X"`,
-    `"Y"`, `"Z"` and `"R"`, with optional prefix `"+"` or `"-"` still work, but now
-    directions can be specified as vectors with 3 components. Users will be warned, and
-    ultimately errored, if the specified directions do not agree with axis directions
-    discovered from the geometry.
-  - Added flux-based error estimation, reported in `error-estimate.csv`. This computes the
-    difference between the numerical gradient (electrostatics) or curl (otherwise) of the
-    solution, and a smoother approximation obtained through a global mass matrix inversion.
-    The results are reported in `error-estimates.csv` within the `"Output"` folder.
-  - Added Adaptive Mesh Refinement (AMR), specified in the `config["Model"]["Refinement"]`,
-    for all problem types aside from transient. To enable AMR, a user must specify
-    `"MaxIts"`, while all other options have reasonable defaults. Nonconformal(all mesh
-    types) and conformal (simplex meshes) refinement are supported.
-  - Added output of lumped port voltage and current for eigenmode simulations.
-  - Added dimensionalized output for energies, voltages, currents, and field values based on
-    a choice of the characteristic magnetic field strength used for nondimensionalization.
-  - Added output of electric and magnetic field energies and participation ratios in regions
-    of the domain, specified with `config["Domains"]["Postprocessing"]["Energy"]` and
-    written to `domain-E.csv`. This replaces
-    `config["Domains"]["Postprocessing"]["Dielectric"]` and `domain-Q.csv`.
-  - Fixed bugs for simulations using tetrahedral meshes associated with unexpected mesh
-    toplogy changes during parallel mesh construction.
-  - Added improved `Timer` and `BlockTimer` classes with more timing categories for
-    reporting simulation runtime.
-  - Added build dependencies on [libCEED](https://github.com/CEED/libCEED) and
-    [LIBXSMM](https://github.com/libxsmm/libxsmm) to support operator partial assembly (CPU-
-    based for now).
+  - Added build dependencies on [libCEED](https://github.com/CEED/libCEED), including
+    [LIBXSMM](https://github.com/libxsmm/libxsmm) and [MAGMA](https://icl.utk.edu/magma/)
+    to support CPU- and GPU-based operator partial assembly.
   - Added unit test framework for all integrators based on
     [Catch2](https://github.com/catchorg/Catch2), which also includes some automated
     benchmarking capabilities for operator assembly and application.
   - Added improved OpenMP support in `palace` wrapper script and CI tests.
-  - Added Apptainer/Singularity container build definition for Palace.
+  - Added Apptainer/Singularity container build definition for *Palace*.
   - Fixed bugs related to thread-safety for OpenMP builds and parallel tetrahedral meshes in
     the upstream MFEM library.
 
@@ -75,7 +59,7 @@ The format of this changelog is based on
     dependencies relying instead directly on CMake's ExternalProject, patch GSLIB dependency
     for shared library builds, add CI tests with ARPACK-NG instead of SLEPc, update all
     dependency versions including MFEM to incorporate bug fixes and improvements. This
-    affects the Spack package recipe, so a new recipe is distributed as part of Palace in
+    affects the Spack package recipe, so a new recipe is distributed as part of *Palace* in
     in `spack/` which will keep compatibility with `main` while changes are upstreamed to
     the built-in Spack repository.
   - Added a basic Makefile with some useful targets for development.

README.md

@@ -12,7 +12,8 @@ SPDX-License-Identifier: Apache-2.0
 *Palace*, for **PA**rallel **LA**rge-scale **C**omputational **E**lectromagnetics, is an
 open-source, parallel finite element code for full-wave 3D electromagnetic simulations in
 the frequency or time domain, using the
-[MFEM finite element discretization library](http://mfem.org).
+[MFEM finite element discretization library](http://mfem.org) and
+[libCEED library](https://github.com/CEED/libCEED) for efficient exascale discretizations.
 
 ## Key features
 
@@ -28,15 +29,13 @@ the frequency or time domain, using the
     problem formulations.
   - Support for a wide range of mesh file formats for structured and unstructured meshes,
     with built-in uniform or region-based parallel mesh refinement.
-  - Solution-based Adaptive Mesh Refinement (AMR) for all simulation types aside from
-    transient. Nonconformal refinement is supported for all mesh types, and conformal
-    refinement for simplex meshes.
   - Arbitrary high-order finite element spaces and curvilinear mesh support thanks to the
     [MFEM library](https://mfem.org/features/).
-  - Scalable algorithms for the solution of linear systems of equations, including geometric
-    multigrid (GMG), parallel sparse direct solvers, and algebraic multigrid
-    (AMG) preconditioners, for fast performance on platforms ranging from laptops to HPC
-    systems.
+  - Scalable algorithms for the solution of linear systems of equations, including matrix-
+    free $p$-multigrid utilizing
+    [high-order operator partial assembly](https://mfem.org/performance/), parallel sparse
+    direct solvers, and algebraic multigrid(AMG) preconditioners, for fast performance on
+    platforms ranging from laptops to HPC systems.
 
 ## Getting started
 
@@ -55,9 +54,9 @@ connection is required.
 
 System requirements:
 
-  - CMake version 3.18.1 or later
+  - CMake version 3.21 or later
   - C++17 compatible C++ compiler
-  - C and (optionally) Fortran compilers for dependency builds
+  - C and Fortran (optional) compilers for dependency builds
   - MPI distribution
   - BLAS, LAPACK libraries
 

docs/src/config/boundaries.md

@@ -104,8 +104,8 @@ frequency domain driven simulation type.
 
 `"WavePortPEC"` :  Top-level object for configuring PEC boundary conditions for boundary
 mode analysis performed on the wave port boundaries. Thus, this object is only relevant
-when wave port boundaries are specified under [`config["Boundaries"]["WavePort"]`]
-(#boundaries[%5B%22WavePort%22%5D]).
+when wave port boundaries are specified under
+[`config["Boundaries"]["WavePort"]`](#boundaries%5B%22WavePort%22%5D).
 
 `"SurfaceCurrent"` :  Array of objects for configuring surface current boundary conditions.
 This boundary prescribes a unit source surface current excitation on the given boundary in
@@ -378,8 +378,9 @@ port boundary, specified in mesh length units.
 with
 
 `"Attributes" [None]` :  Integer array of mesh boundary attributes to consider along with
-those specified under [`config["Boundaries"]["PEC"]["Attributes"]`]
-(#boundaries%5B%22PEC%22%5D) as PEC when performing wave port boundary mode analysis.
+those specified under
+[`config["Boundaries"]["PEC"]["Attributes"]`](#boundaries%5B%22PEC%22%5D) as PEC when
+performing wave port boundary mode analysis.
 
 ## `boundaries["SurfaceCurrent"]`
 
@@ -438,9 +439,9 @@ element of a multielement current source can be described by its own unique dire
 which is specified here. The elements of a multielement source add in parallel to give the
 same total current as a single-element source.
 
-`"Elements"[]["CoordinateSystem"] ["Cartesian"]` :  This option is for multielement surface current
-boundaries and should not be combined with the `"CoordinateSystem"` field described above. Each
-element of a multielement current source can be described by its own unique
+`"Elements"[]["CoordinateSystem"] ["Cartesian"]` :  This option is for multielement surface
+current boundaries and should not be combined with the `"CoordinateSystem"` field described
+above. Each element of a multielement current source can be described by its own unique
 direction, and corresponding coordinate system.
 
 ## `boundaries["Ground"]`

docs/src/config/model.md

@@ -83,7 +83,8 @@ per iteration, at the cost of the final mesh being less efficient.
 
 `"UniformLevels" [0]` :  Levels of uniform parallel mesh refinement to be performed on the
 input mesh. If not performing AMR, these may be used as levels within a geometric multigrid
-scheme.
+scheme. If performing AMR the most refined mesh is used as the initial mesh and the coarser
+meshes cannot be used in a geometric multigrid scheme.
 
 `"Boxes"` :  Array of box region refinement objects. All elements with a node inside the box
 region will be marked for refinement.