Updating documentation for release

.binder/apt.txt

@@ -6,4 +6,3 @@ libxinerama1
 libfltk1.3-dev
 libfreetype6-dev
 libgl1-mesa-dev
-python3-gmsh

.binder/environment.yml

@@ -2,16 +2,18 @@ name: uw3
 channels:
   - conda-forge
 dependencies:
-  - python=3.10
-  - petsc=3.21.5
+  - python=3.11
+  - compilers
+  - openmpi
+  - openmpi-mpicc
+  - openmpi-mpicxx
+  - petsc=3.21.5 # Fix for now (errors with hdf5 / uw3)
   - petsc4py=3.21.5
   - cython=3.*
   - mpmath<=1.3
   - mesalib
   - numpy
   - scipy
-  - mpich-mpicc
-  - mpich-mpicxx
   - mpi4py
   - h5py
   - sympy

.github/.devcontainer/uw_environment.yml

@@ -2,28 +2,30 @@ name: uw3
 channels:
   - conda-forge
 dependencies:
-  - petsc=3.21.0
-  - petsc4py=3.21.0
+  - python=3.11
+  - compilers
+  - openmpi
+  - openmpi-mpicc
+  - openmpi-mpicxx
+  - petsc=3.21.5 # Fix for now (errors with hdf5 / uw3)
+  - petsc4py=3.21.5
   - cython=3.*
   - mpmath<=1.3
   - mesalib
   - numpy
   - scipy
-  - mpich-mpicc
-  - mpich-mpicxx
   - mpi4py
   - h5py
   - sympy
   - pytest
-  - ipython 
+  - ipython
   - pyvista
-  - quarto
   - pip
-  - pip:  
-    - gmsh
-    - jupyterlab>=4.1
-    - jupytext>=1.16.1
-    - pygments>=2.17.0
-    - trame-vtk
-    - trame-vuetify
-
+  - pip:
+      - gmsh
+      - gmsh-api
+      - jupyterlab
+      - jupytext
+      - pygments
+      - trame-vtk
+      - trame-vuetify

.github/CONTRIBUTING.md

@@ -1,8 +1,7 @@
 This page provides information about contributing to Underworld’s codebase.
+For contributions of Underworld models please go https://github.com/underworld-community
 
-For contributions to Underworld models please go https://github.com/underworld-community
-
----- 
+----
 
 We welcome contributions to Underworld’s codebase in the form of:
 
@@ -11,8 +10,8 @@ We welcome contributions to Underworld’s codebase in the form of:
   * Suggestions / Requests
   * Documentation modifications (including docstrings)
 
-For Bug reports and Suggestions / Requests please submit an Issue on the Underworld GitHub Issue Tracker. 
-Please tag the Issue with a given Label to help us assess the issue and provide simple scripts that explain how to 
+For Bug reports and Suggestions / Requests please submit an Issue on the Underworld GitHub Issue Tracker.
+Please tag the Issue with a given Label to help us assess the issue and provide simple scripts that explain how to
 reproduce the problem.
 
 Click here to submit an Issue https://github.com/underworldcode/underworld3/issues
@@ -30,14 +29,14 @@ More specifically:
 2. Add the master Underworld repository as an additional remote source (named `uwmaster`) for your local repo and pull down its latest changesets. Checkout to the master/development repo state, and then create a new local branch which will contain your forthcoming changes.
 
 ``` bash
-  
+
     git remote add uw3 https://github.com/underworldcode/underworld3
     git pull uw3
     git checkout uw3/development
     git checkout -b newFeature
 
 ```
-     
+
 3. Make your changes! Remember to write comments, a test if applicable and follow the code style of the project<!-- (see `./docs/development/guidelines.md` for details). NB: this is on the todo list for uw3 -->
 
-4. Push your changes to your GitHub fork and then submit a PR to the `development` branch of Underworld via Github.
+4. Push your changes to your GitHub fork and then submit a PR to the `development` branch of Underworld via Github.

.github/workflows/draft-pdf.yml

@@ -1,12 +1,12 @@
-name: Draft PDF
-on: 
-    push:
-        paths:
-            - joss-paper/paper.md
-            - joss-paper/paper.bib
-            # - assets/img1.png
-            # - assets/img2.png
-            - .github/workflows/draft-pdf.yml
+name: Draft PDF (JOSS)
+on:
+  push:
+    paths:
+      - joss-paper/paper.md
+      - joss-paper/paper.bib
+      # - assets/img1.png
+      # - assets/img2.png
+      - .github/workflows/draft-pdf.yml
 
 jobs:
   paper:
@@ -28,4 +28,4 @@ jobs:
           # This is the output path where Pandoc will write the compiled
           # PDF. Note, this should be the same directory as the input
           # paper.md. In the case we have (symlink), the paper is in the subdir
-          path: joss-paper/paper.pdf
+          path: joss-paper/paper.pdf

.zenodo.json

@@ -20,9 +20,14 @@
             "orcid": "0000-0003-0817-354X",
             "affiliation": "Research School of Earth Sciences, The Australian National University",
         },
+        {
+            "name": "Matt Knepley",
+            "orcid": "0000-0002-2292-0735"
+            "affiliation": "Computer Science and Engineering, University at Buffalo",
+        },
         {
             "name": "Ben Knight",
-            "affiliation": "Monash University",
+            "affiliation": "School of Earth, Atmospheric & Environmental Science, Monash University",
             "orcid": "0000-0001-7919-2575"
         },
         {
@@ -32,7 +37,7 @@
         },
         {
             "name": "John Mansour",
-            "affiliation": "TBD",
+            "affiliation": "School of Earth, Atmospheric & Environmental Science, Monash University",
             "orcid": "0000-0001-5865-1664"
         },
         {
@@ -45,4 +50,4 @@
     "title": "Underworld3: Mathematically Self-Describing Modelling in Python for Desktop, HPC and Cloud",
     "upload_type": "software",
     "access_right": "open"
-}
+}

LICENCE.md

@@ -1,12 +1,11 @@
 ### Summary
 
 Underworld is an open-source, parallel, particle-in-cell, finite element geodynamics code. Please refer to repository
-top level `README.md` for further information. 
+top level `README.md` for further information.
 
 ### Licensing
 
-1) All Underworld source code is released under the LGPL-3 (See LGPLv3.txt in this repository). This covers all
-files in `underworld` constituting the Underworld3 Python module (library), and any other material not explicitly identified under (2) below.
+1) All Underworld source code is released under the LGPL-3 (See the file LICENCE in his repository). This covers all files in `underworld` constituting the Underworld3 Python module (library), and any other material not explicitly identified under (2) below.
 
 2) Notebooks, stand-alone documentation and Python scripts which show how the code is used and run are licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.  We offer this licence to encourage you to modify and share the examples and use them to help you in your research. Where no individual creator is identified in these files, the appropriate attribution is "The Underworld Team". All the files covered by this license are found in the `UserGuide` directory.
 
@@ -32,7 +31,3 @@ Copyright VPAC,                           2003-2009
    1. Beucher, R., Moresi, L., Giordani, J., Mansour, J., Sandiford, D., Farrington, R., et al. (2019). UWGeodynamics: A teaching and research tool for numerical geodynamic modelling. Journal of Open Source Software. https://doi.org/10.21105/joss.01136
 
    1. Mansour, J., Giordani, J., Moresi, L., Beucher, R., Kaluza, O., Velic, M., et al. (2020). Underworld2: Python Geodynamics Modelling for Desktop, HPC and Cloud. Journal of Open Source Software, 5(47), 1797. https://doi.org/10.21105/joss.01797
-
-
-
-

docs/user/Installation.qmd

@@ -8,9 +8,84 @@ exports:
 - format: html
 ---
 
+## Source code build using mamba
 
+We recommend that you use a `conda` / `mamba` virtual environment to build `underworld3` whenever you want to install on a personal workstation or laptop (linux or macOS).The [`mamba` documentation](https://mamba.readthedocs.io/en/latest/index.html) will help you get started if you are not familiar with the philososphy and practice of this package management system.
 
+The `underworld3` build / run-time dependencies can be installed using
 
+```bash
+    (base) % git clone -b development --single-branch https://github.com/underworldcode/underworld3 /path/to/underworld3
+    (base) % cd /path/to/underworld3
+    (base) % mamba env create -n uw3 -f environment.yml
+    (base) % mamba activate uw3
 
+    (uw3) % pip install .
+```
 
-Windows users, if you don't want to create a linux partition, you can use our containers.
+## Source code build using mamba and existing PETSc
+
+If you have an exisiting build of `PETSc` that you need to use, the following
+instructions work well. We create the environment as before but remove the pre-
+installed `PETSc` and `petsc4py`.
+
+```bash
+    (base) % git clone -b development --single-branch https://github.com/underworldcode/underworld3 /path/to/underworld3
+    (base) % cd /path/to/underworld3
+    (base) % mamba env create -n uw3p -f environment.yml
+    (base) % mamba activate uw3p
+
+    (uw3p) % mamba remove petsc4py
+    (uw3p) % mamba remove petsc
+```
+
+Next set up PETSc as you like it and build it using the tools
+within the current mamba virtual environment. Set the `PETSC_ARCH`
+environment variable to the name of this virtual environment to keep things
+from becoming muddled. See [PETSc Installation]() for details on
+how to configure and build what you need.
+
+```bash
+
+    (uw3p) % export PETSC_DIR=/path/to/petsc
+    (uw3p) % export PETSC_ARCH="uw3p"
+    (uw3p) % git clone -b release https://gitlab.com/petsc/petsc.git $PETSC_DIR
+    (uw3p) % cd $PETSC_DIR
+    (uw3p) % # Configure & Build step
+```
+Next we `pip install` the petsc4py that we just built into the mamba virtual environment. This
+ensures that `petsc4py` is available and also that its internal configuration points to the `PETSc` installation we just created. This way we don't need to manage environment variables to point to the build that corresponds to a given virtual environment.
+
+```bash
+    (uw3p) % mamba install cmake
+    (uw3p) % cd $PETSC_DIR/src/binding/petsc4py
+    (uw3p) % pip install .
+```
+
+Building `underworld3` against this version of `PETSc` should be identical to the mamba version.
+
+```bash
+    (uw3p) % cd /path/to/underworld3
+    (uw3p) % pip install .
+```
+
+#### Did it work ?
+
+First run the tests:
+
+```bash
+    (uw3p) % cd /path/to/underworld3
+    (uw3p) % source test.sh
+```
+
+If you have problems, contact us via the `underworl3` [GitHub issue tracker](https://github.com/underworldcode/underworld3/issues)
+
+## Docker container
+
+Windows users, if you don't want to create a linux partition, you can use our containers with `docker` or `podman`. As the code is still in active development, we are not always able to provide containers for each change to the development branch. We ask that you reach out to us on our [GitHub issue tracker](https://github.com/underworldcode/underworld3/issues) to ask for information (use the "question" label).
+
+## HPC builds
+
+`Underworld3` is inherently parallel and designed for use in high performance computing environments. The symbolic layer is relatively lightweight and should not adversely affect launch time (drawing down libraries from disk) or execution time (very few calculations are done in python itself).
+
+In the HPC environment you may find it difficult to control the software stack. You will need to ensure that you can build against PETSc 3.21 or higher, since important functionality that we need is only available from that release onwards. We are happy to provide assistance with builds on specific machines and ask that you contact us through the [GitHub issue tracker](https://github.com/underworldcode/underworld3/issues) so that other people are able to browse the issues and their fixes.

docs/user/NextSteps.qmd

@@ -12,31 +12,86 @@ exports:
 
 ### Underworld Documentation and Examples
 
+In addition to the notebooks in this brief set of examples, there are a number of sources of information on using `Underworld3` that you can access:
+
+  - [The Underworld Website / Blog](https://www.underworldcode.org)
+
+  - [The API documentation](https://underworldcode.github.io/underworld3/development_api/index.html)
+  (all the modules and functions and their full sets of arguments) is automatically generated from the source code and uses the same rich markdown content as the notebook help text.
+
+  - The [`underworld3` GitHub repository](https://github.com/underworldcode/underworld3) is the most active development community for the code.
+
 
 ### Benchmarks
 
+The [Underworld3 Benchmarks Repository](https://github.com/underworld-community/UW3-benchmarks) is a useful place to find community benchmarks coded in `underworld3` along with accuracy and convergence analysis. This is an open repository where you can make a pull request with new benchmark submissions once you get the hang of things.
 
 ### The Underworld Community
 
-Sharing codes / fixes
-
-Sharing models
+The [Underworld Community](https://github.com/underworld-community) organisation on Github is a collection of contributed repositories from the underworld user community for all versions of the code and now includes scripts for underworld3.
 
 ### Parallel Execution
 
-Advise using jupytext to render notebooks at .py files. This is helpful
-in shared environments because the files contain no output and play
-better with version control.
+`Underworld3` is inherently parallel and designed for high performance computing. The symbolic layer is a relatively thin veneer that sits on top of the `PETSc` machinery. A script that has been developed in `jupyter` should be transferrable to an HPC environment with no changes (except that inherently serial operations such as visualisation and model reduction are best left to postprocessing / co-processing).
+
+We recommend installing `jupytext` which allows either a seamless two-way conversion (or pairing) between the `ipynb` format and an annotated python script, or the ability to work entirely with annoted python scripts and not use `ipynb` at all. The python form does not store the output of notebook cells but there are advantages to this when scripts are under version control.
+
+Almost all of our notebook examples are annotated python for this reason.An exception is the collection of notebooks in this quick-start guide because we want to show you the rendered output in the static web pages.
+
+```bash
+    mpirun -np 1024 python3 Modelling_Script.py -uw_resolution 96
+```
+
+The main difference between the notebook development environment and HPC is the lack of interactivity, particularly in sending parameters to the script at launch time. Typically, we expect the HPC version to be running at much higher resolution, or for many more timesteps than the development notebook. We use the `PETSc` command line parsing machinery to generate notebooks that also can ingest run-time parameters from a script (as above).
+
+
+### Advanced capabilities
+
+Digging a bit deeper into `underworld3`, there are many capabilities that require a clear understanding of the concepts that underlie the implementation. The following examples are not *plug-and-play*, but they do only require python coding using the `underworld3` API and no detailed knowledge of `petsc4py` or `PETSc`. [Get in touch with us](https://github.com/underworldcode/underworld3/issues) if you want to try this sort of thing but can't figure it out for yourself.
 
-Not such a good idea if you want to have a tutorial with outputs (like this one)
+#### Deforming meshes
+
+In [Example 8](Notebooks/8-Particle_Swarms.ipynb), we made small variations to the mesh to conform to basal topography. We did not remesh, so we had to be careful to apply a smooth, analytic displacement to every node. For more general free-surface models, we need to calculate a smooth function using the computed boundary motions (e.g, solving a poisson equation with known boundary displacements as boundary conditions). We need to step through time and it is common to stabilize the surface motions through a surface integral term that accounts for the interface displacement during the timestep. The example below shows an `underworld3` forward model with internal loads timestepped close to isostatic equilibrium.
+
+![*Stokes flow driven by buoyancy in an annulus defined by two embedded surfaces within an enveloping disk mesh. The surfaces deform in response to the flow. The embedding medium has a very low viscosity but still acts to damp rotational modes. The outer boundary of the disk can be set to a far-field gravitational potential for whole-Earth relaxation models*](media/RelaxingMesh.png){width=50%}
+
+In a more general case, we need to account to horizontal motions. This is more complicated because the horizontal velocities can be large even when vertical equilibrium is achieved. So we need to solve for the advected component of vertical motion in addition to the local component. Hooray for symbolic algebra !
+
+#### Weak / penalty boundary conditions
+
+[Example 8](Notebooks/8-Particle_Swarms.ipynb) introduced the idea of penalty-based boundary conditions where the constraint is weakly enforced by providing a large penalty for violation of the condition. This is very flexible as the penalizing conditions can be adjusted during the run, including changing which part of the boundary is subject to constraints based on the solution or a coupled problem. The channel flow model shown below has a boundary condition that depends on a externally sourced model for ponded material at the base that is derived from a simple topography filling algorithm.
+
+```{=html}
+<center>
+<iframe src="media/pyvista/ChannelFlow.html" width="600" height="300">
+</iframe>
+</center>
+```
+*Live Image: Stokes flow in a channel with multiple obstructions. Flow is driven from the inlet (a velocity boundary condition). The geometry was contructed with `gmsh`. This is an example for education which demonstrates the emergence of an large-scale pressure gradient as a result of the presence of the obstructions, and also the dispersion of tracers through the complicated flow geometry*
+
+The penalty approach does allow the solution to deviate from the exact value of the boundary condition, in a similar way to the iterative solvers delivering a solution to within a specified tolerance. There are some cases, for example, enforcing plate motions at the surface, where there are uncertainties in the applied boundary conditions and that these uncertainties may vary in space and time.
+
+#### Mesh Adaptation
+
+It is also possible to use the PETSc mesh adaption capabilities, to refine the resolution of a model where it is most needed and coarsen elsewhere. Dynamically-adapting meshes are possible but the interface is very low level at present, particularly in parallel.
+
+```{=html}
+<center>
+<iframe src="media/pyvista/AdaptedSphere.html" width="600" height="300">
+</iframe>
+</center>
+```
+*Live Image: Static mesh adaptation to the slope of a field. The driving buoyancy term is three plume-like upwellings and the slope of this field is shown in colour (red high, blue low). The adapted mesh is shown in green.*
 
+```python
 
+    # t is the driving "temperature". We form an isotropic refinement metric from its slope
 
-<!--
-```{iframe} https://github.com/underworldcode/underworld3/raw/VEP-preliminary-implementation/Documentation/media/Sinker.html
-:width: 100%
-:align: center
+    refinement_fn = 1.0 + sympy.sqrt(
+          t.diff(x) ** 2
+        + t.diff(y) ** 2
+        + t.diff(z) ** 2
+    )
 
-Caption
+    icoord, meshA = adaptivity.mesh_adapt_meshVar(mesh0, refinement_fn, Metric, redistribute=True)
 ```
--->