Skip to content

Commit

Permalink
UW-501 / UW-502 FV3 graph-based driver implementation (#408)
Browse files Browse the repository at this point in the history
  • Loading branch information
maddenp-noaa authored Feb 14, 2024
1 parent 0e75475 commit 666880f
Show file tree
Hide file tree
Showing 49 changed files with 2,385 additions and 2,402 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ TARGETS = clean-devenv devshell docs env format lint meta package test typec

export RECIPE_DIR := $(shell cd ./recipe && pwd)

clean = $(shell $(CONDA_EXE) env remove -n DEV-$(call val,name))
clean = $(info $(shell $(CONDA_EXE) env remove -n DEV-$(call val,name)))
spec = $(call val,name)$(2)$(call val,version)$(2)$(call val,$(1))
val = $(shell jq -r .$(1) $(METAJSON))

Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
with open("../recipe/meta.json", "r", encoding="utf-8") as f:
_metadata = json.loads(f.read())

autodoc_mock_imports = ["f90nml", "jsonschema", "lxml"]
autodoc_mock_imports = ["f90nml", "iotaa", "jsonschema", "lxml"]
copyright = str(dt.datetime.now().year)
extensions = ["sphinx.ext.autodoc", "sphinx.ext.extlinks", "sphinx.ext.intersphinx"]
extlinks_detect_hardcoded_links = True
Expand Down
14 changes: 6 additions & 8 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -96,17 +96,15 @@ Do you already have a Rocoto XML but don't want to run Rocoto to make sure it wo
The Drivers
-----------

The uwtools driver(s) live right there beside the rest of the tools in the CLI and API. These tools will be under development for the foreseeable future, but we do have a forecast driver currently available in beta testing mode.
Drivers for NWP components are available as top-level CLI modes and API modules.

Forecast
^^^^^^^^
FV3
^^^

| **CLI**: ``uw forecast -h``
| **API**: ``import uwtools.api.drivers.forecast``
| **CLI**: ``uw fv3 -h``
| **API**: ``import uwtools.api.drivers.fv3``
This driver is the first of its kind (with many others to come) and takes a few pieces of information from the user --- the model, the time, and a structured YAML --- and runs a forecast via batch job or as an executable. That simple.

We've helped by providing a JSON Schema that allows you to validate your YAML to ensure you've got it right!
Provided with a valid UW YAML configuration file and a forecast-cycle value, ``uw fv3`` can prepare a fully provisioned FV3 run directory, execute FV3 directly, or can submit an FV3 batch job to an HPC scheduler.

Over time, we'll add many other drivers to support a variety of UFS components from pre-processing to post-processing, along with many data assimilation components.

Expand Down
5 changes: 0 additions & 5 deletions docs/sections/user_guide/api/forecast.rst

This file was deleted.

5 changes: 5 additions & 0 deletions docs/sections/user_guide/api/fv3.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
``uwtools.api.fv3``
========================

.. automodule:: uwtools.api.fv3
:members:
2 changes: 1 addition & 1 deletion docs/sections/user_guide/api/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ API

.. toctree::
config
forecast
fv3
logging
rocoto
template
2 changes: 1 addition & 1 deletion docs/sections/user_guide/cli/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ CLI
:maxdepth: 1

mode_config
mode_forecast
mode_fv3
mode_rocoto
mode_template
4 changes: 2 additions & 2 deletions docs/sections/user_guide/cli/mode_config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The ``uw`` mode for handling configuration files (configs).
``compare``
-----------

The ``compare`` mode lets users compare two config files.
The ``compare`` mode lets users compare two config files.

.. code-block:: text
Expand Down Expand Up @@ -148,7 +148,7 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``,
``realize``
-----------

In ``uw`` terminology, to realize a configuration file is to transform it from its raw form into its final, usable state. The ``realize`` mode can build a complete config file from two or more separate files.
In ``uw`` terminology, to realize a configuration file is to transform it from its raw form into its final, usable state. The ``realize`` mode can build a complete config file from two or more separate files.

.. code-block:: text
Expand Down
88 changes: 0 additions & 88 deletions docs/sections/user_guide/cli/mode_forecast.rst

This file was deleted.

103 changes: 103 additions & 0 deletions docs/sections/user_guide/cli/mode_fv3.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
Mode ``fv3``
============

The ``uw`` mode for configuring and running FV3.

.. code-block:: text
$ uw fv3 --help
usage: uw fv3 [-h] TASK ...
Execute FV3 tasks
Optional arguments:
-h, --help
Show help and exit
Positional arguments:
TASK
boundary_files
The FV3 lateral boundary-condition files
diag_table
The FV3 diag_table file
field_table
The FV3 field_table file
files_copied
Files copied for FV3 run
files_linked
Files linked for FV3 run
model_configure
The FV3 model_configure file
namelist_file
The FV3 namelist file
provisioned_run_directory
The run directory provisioned with all required content
restart_directory
The FV3 RESTART directory
run
FV3 run execution
runscript
A runscript suitable for submission to the scheduler
All tasks take the same arguments. For example:

.. code-block:: text
$ uw fv3 run --help
usage: uw fv3 run --config-file PATH --cycle CYCLE [-h] [--batch] [--dry-run] [--debug] [--quiet]
[--verbose]
FV3 run execution
Required arguments:
--config-file PATH, -c PATH
Path to config file
--cycle CYCLE
The cycle in ISO8601 format
Optional arguments:
-h, --help
Show help and exit
--batch
Submit run to batch scheduler
--dry-run
Only log info, making no changes
--debug
Print all log messages, plus any unhandled exception's stack trace (implies --verbose)
--quiet, -q
Print no logging messages
--verbose, -v
Print all logging messages
Examples
^^^^^^^^

The examples use a configuration file named ``config.yaml``. Its contents are described in depth in section :ref:`fv3_yaml`.

* Run FV3 on an interactive node

.. code-block:: text
$ uw fv3 run --config-file config.yaml --cycle 2024-02-11T12
The driver creates a ``runscript`` file in the directory specified by ``run_dir`` in the config and runs it, executing FV3.

* Run FV3 via a batch job

.. code-block:: text
$ uw fv3 run --config-file config.yaml --cycle 2024-02-11T12 --batch
The driver creates a ``runscript`` file in the directory specified by ``run_dir`` in the config and submits it to the batch system. Running with ``--batch`` requires a correctly configured ``platform`` block in ``config,yaml``, as well as appropriate settings in the ``execution`` block under ``fv3``.

* Specifying the ``--dry-run`` flag results in the driver logging messages about actions it would have taken, without actually taking any.

.. code-block:: text
$ uw fv3 run --config-file config.yaml --cycle 2024-02-11T12 --batch --dry-run
* The ``run`` task depends on the other available tasks and executes them as prerequisites. It is possible to execute any task directly, which entails execution of any of *its* dependencies. For example, to create an FV3 run directory provisioned with all the files, directories, symlinks, etc. required per the configuration file:

.. code-block: text
$ uw fv3 provisioned_run_directory --config-file config.yaml --cycle 2024-02-11T12 --batch
4 changes: 2 additions & 2 deletions docs/sections/user_guide/installation.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Installation
============

.. note::
.. note::

Developers should visit the :doc:`Developer Setup <../contributor_guide/developer_setup>` section located in the :doc:`Contributor Guide <../contributor_guide/index>`.

The recommended installation mechanism uses the Python package and virtual-environment manager :conda:`conda<>`. Specifically, these instructions assume use of the :miniforge:`Miniforge<>` variant of :miniconda:`Miniconda<>`, built to use, by default, packages from the :conda-forge:`conda-forge<>` project. Users of the original :miniconda:`Miniconda<>` or the :anaconda:`Anaconda distribution<>` should add the flags ``-c conda-forge --override-channels`` to ``conda`` commands to specify the required package channels.
Expand Down
Loading

0 comments on commit 666880f

Please sign in to comment.