From 2e7b80ee8fc9abdfcbc1578520abba29d600853e Mon Sep 17 00:00:00 2001 From: Luca Bertagna Date: Mon, 29 Apr 2024 16:45:02 -0600 Subject: [PATCH 1/6] EAMxx: add documentation for model inputs --- components/eamxx/docs/user/model_input.md | 267 +++++++++++++++++++++- 1 file changed, 264 insertions(+), 3 deletions(-) diff --git a/components/eamxx/docs/user/model_input.md b/components/eamxx/docs/user/model_input.md index 38486c77a21..702bf7c89b7 100644 --- a/components/eamxx/docs/user/model_input.md +++ b/components/eamxx/docs/user/model_input.md @@ -1,8 +1,269 @@ -Model input +Model inputs ===================================== -TODO: explain how defaults XML, atmchange/atmquery, buildml, and input.yaml work. +This section explains how input parameters are passed to EAMxx, and how the user can +change their value. The full list of the currently configuraable runtime parameters for +EAMxx can be found [here](../common/eamxx_params.md). -[Here](../common/eamxx_params.md) is a list of the currently configurable runtime parameters for EAMxx. +The main files/scripts involved in the EAMxx runtime configuration are the following: +1. `buildnml`: this script, located in `components/eamxx/cime_config`, is called by CIME's case management + scripts (`case.setup`, `case.build, `case.submit`), and is responsible for creating EAMxx input files, + including handling of user-specific modifications for the runtime parameters. Users should not have to + modify this script, nor should they have to manually call it. +2. `scream_input.yaml`: EAMxx will parse this [YAML](https://yaml.org/spec/1.2.2) file at runtime, + to detect all of its configuration parameters. It is generated automatically by `buildnml` inside the + RUNDIR/data folder, where `RUNDIR` is the run directory generated by CIME. Since this file is automatically + generated when `buildnml` runs, users should not manually modify it. Any manual modification will be + lost the next time `buildnml` runs (e.g., at `case.submit` time). +3. 'namelist_scream.xml`: this XML file is generated by `buildnml`, and contains all the runtime parameters that EAMxx + will read in at runtime. `buildnml` uses this XML file as an intermediate file during the generation of + `scream_input.yaml`. In particular, `buildnml` generates this file using case information to select the + proper configuration from the file `namelist_defaults_scream.xml`, located in `components/eamxx/cime_config`. + As it was the case for `scream_input.yaml`, since this XML file is automatically generated by `buildnml`, + the user should not manually modify this file. Any manual modification will be lost the next time `buildnml` + runs (e.g., at `case.submit` time). + Since the main product of `buildnml` is `scream_input.yaml`, one may wonder why we keep + this XML file around at all. The main reason is to make it easier for the `atmquery` script (see below) to + retrieve settings. +4. `namelist.nl`: EAMxx will parse this [namelist](https://docs.oracle.com/cd/E19957-01/805-4939/6j4m0vnad/index.html) + file at runtime, to retrieve all the parameters specific to the [Homme](some_homme_ref) dycore. + It is generated automatically by `buildnml` inside the RUNDIR/data folder, where `RUNDIR` is the run + directory generated by CIME. Since this file is automatically generated when `buildnml` runs, + users should not manually modify it. Any manual modification will be lost the next time `buildnml` runs + (e.g., at `case.submit` time). +5. `atmchange` and `atmquery`: these script, located in `components/eamxx/scripts` and linked in the case folder, + can be used to query and change any of the runtime configuration parameters of EAMxx. +Among the files listed above, `atmquery` and `atmchange` are the only ones that the user should interact with, +and therefore we give here a brief overview of how they can be used, and how their output can be interpreted. +For both these scripts, a short help can also be obtained using the `-h` flag, highlighting their use cases and options. + +# Querying parameters with `atmquery` + +This script is the simplest way for the user to check the value and properties of EAMxx input parameters. +A basic usage of the script is + +```bash +$ ./atmquery my_param +``` +which will retrieve the value of the parameter called `my_param`, by locating the XML node "my_param" in the +file `namelist_scream.xml` in the RUNDIR folder. Obviously, an XML file can have multiple nodes with the same tag, +and the script is implemented to error out if multiple matches are found. In such a scenario, the user needs +to provide also the parents nodes names, using enough parents to uniquely identify the node (in most cases, +one parent is enough). To specify a parent, the user can prepend the parent name and `::` to the node name: + +```bash +$ ./atmquery foo::my_param +``` + +The output will contain the fully scoped parameter name, along with the value. E.g., +```bash +$ ./atmquery foo::my_param + namelist_defaults::node1::node2::foo::my_param: 10 +``` + +It is sometimes desirable to query _all_ the nodes that have a particular name, or that contain a particular +string. We can do that by using the `--grep` flag: +```blah +$ ./atmquery --grep sub + iop_options::iop_dosubsidence: false + ctl_nl::hypervis_subcycle: 1 + ctl_nl::hypervis_subcycle_tom: 1 + ctl_nl::hypervis_subcycle_q: 6 + atmosphere_processes::number_of_subcycles: 1 + sc_import::number_of_subcycles: 1 + homme::number_of_subcycles: 1 + physics::number_of_subcycles: 1 +``` +TODO: This difference between basica and `--grep` is not really intuitive (see [this +issue](https://github.com/E3SM-Project/scream/issues/2413)). +Using the `--grep` option has another effect: if the match is not a leaf of the XML tree, all its subelements +are printed: +```bash +$ ./atmquery --grep homme + homme + Moisture: moist + BfbHash: 18 + number_of_subcycles: 1 + enable_precondition_checks: true + enable_postcondition_checks: true + repair_log_level: trace + internal_diagnostics_level: 0 + compute_tendencies: None +``` + +Similarly to the CIME utility `xmlchange`, the options `--value`, `--type`, `--valid-values`, and `--full` can be +used to respectively retrieve just the parameter value (useful for bash scripting), the parameter's type, +a list of valid values for parameter (when applicable), or all of the above: +```bash +$ ./atmquery atm_log_level --value + info +$ ./atmquery atm_log_level --type + namelist_defaults::driver_options::atm_log_level: string +$ ./atmquery atm_log_level --valid-values + namelist_defaults::driver_options::atm_log_level: ['trace', 'debug', 'info', 'warn', 'error'] +$ ./atmquery atm_log_level --full + namelist_defaults::driver_options::atm_log_level + value: info + type: string + valid values: ['trace', 'debug', 'info', 'warn', 'error'] +``` + +Finally, the option `--listall` can be used to list the whole content of the XML file, which will be displayed +with each node indented in its parent scope: +```bash +$ ./atmquery --listall + namelist_defaults + grids_manager + Type: Homme + physics_grid_type: PG2 + physics_grid_rebalance: None + dynamics_namelist_file_name: ./data/namelist.nl + vertical_coordinate_filename: /some/path/to/coords/file.nc + initial_conditions + Filename: /some/path/to/ic/file.nc + topography_filename: /some/path/to/topo/file.nc + ... +``` + +# Changing model inputs via `atmchange` + +When `buildnml` runs, the model inputs are deduced from the case configuration settings (e.g., the grid, +the compset, etc.) and the `namelist_scream_defaults.xml` file, located in the eamxx source tree. +The user can change any of these parameters using the `atmchange` script. +A basic usage of the script is + +```bash +$ ./atmchange my_param=10 +``` +As for `atmquery`, if there are multiple matches for a given parameter name, the user must specify a unique +scoped name, which allows `atmchange` to uniquely identify the XML node to modify: +```bash +$ ./atmquery homme::number_of_subcycles + namelist_defaults::atmosphere_processes::homme::number_of_subcycles: 1 +$ ./atmchange number_of_subcycles=10 +ERROR: number_of_subcycles is ambiguous (use --all to change all matches), matches: + namelist_defaults::atmosphere_processes::number_of_subcycles + namelist_defaults::atmosphere_processes::sc_import::number_of_subcycles + namelist_defaults::atmosphere_processes::homme::number_of_subcycles + namelist_defaults::atmosphere_processes::physics::number_of_subcycles + namelist_defaults::atmosphere_processes::physics::mac_aero_mic::number_of_subcycles + namelist_defaults::atmosphere_processes::physics::mac_aero_mic::tms::number_of_subcycles + namelist_defaults::atmosphere_processes::physics::mac_aero_mic::shoc::number_of_subcycles + namelist_defaults::atmosphere_processes::physics::mac_aero_mic::cldFraction::number_of_subcycles + namelist_defaults::atmosphere_processes::physics::mac_aero_mic::p3::number_of_subcycles + namelist_defaults::atmosphere_processes::physics::rrtmgp::number_of_subcycles + namelist_defaults::atmosphere_processes::sc_export::number_of_subcycles +$ ./atmchange homme::number_of_subcycles=10 +Regenerating /path/to/namelist_scream.xml. Manual edits will be lost. +$ ./atmquery homme::number_of_subcycles + namelist_defaults::atmosphere_processes::homme::number_of_subcycles: 10 +``` +In some cases, the user may be interested in changing _all_ nodes with a given name. In that case, +the `--all` flag can be used: +```bash +$ ./atmquery --grep number_of_subcycles + atmosphere_processes::number_of_subcycles: 1 + sc_import::number_of_subcycles: 1 + homme::number_of_subcycles: 1 + physics::number_of_subcycles: 1 + mac_aero_mic::number_of_subcycles: 24 + tms::number_of_subcycles: 1 + shoc::number_of_subcycles: 1 + cldFraction::number_of_subcycles: 1 + spa::number_of_subcycles: 1 + p3::number_of_subcycles: 1 + rrtmgp::number_of_subcycles: 1 + sc_export::number_of_subcycles: 1 +$ ./atmchange --all number_of_subcycles=3 +Regenerating /path/to/namelist_scream.xml. Manual edits will be lost. +$ ./atmquery --grep number_of_subcycles + atmosphere_processes::number_of_subcycles: 3 + sc_import::number_of_subcycles: 3 + homme::number_of_subcycles: 3 + physics::number_of_subcycles: 3 + mac_aero_mic::number_of_subcycles: 3 + tms::number_of_subcycles: 3 + shoc::number_of_subcycles: 3 + cldFraction::number_of_subcycles: 3 + spa::number_of_subcycles: 3 + p3::number_of_subcycles: 3 + rrtmgp::number_of_subcycles: 3 + sc_export::number_of_subcycles: 3 +``` +Since the XML file stores constraints on the parameter value (like its type or valid values), attempting to use the +wrong type will cause an error: +```bash +$ ./atmquery --type se_ne + namelist_defaults::ctl_nl::se_ne: integer +$ ./atmchange se_ne=hello +ERROR: Could not refine 'hello' as type 'integer': +``` +There are three main types supported: integer, float, string, logical. When passing a string to `atmchange`, +the script will try to interpret it acoording to the parameter type, and throw an error if that's not possible: +for "string", anything works; for "integer", only digits are allowed, possibly with a negative sign in front; +for "float", only digits are allowed, possibly with a negative sign in front and a decimal point; for "logical", +only the strings "true" and "false" are allowed (case insensitive). There are two additional types supported: +"file" and "array(T)", where "T" is any of the other supported types (but not another array): + - "file" is used to inform CIME of the input files that have to be download from E3SM data servers, like initial conditions files, +or certain lookup tables. + - "array(T)" allows to specify a list of items (of the same type), which will be parsed inside EAMxx as + a `std::vector`. + +For type "string" and "array(T)", it is also possible to _append_ to the currently stored value +```bash +$ ./atmquery homme::compute_tendencies + namelist_defaults::atmosphere_processes::homme::compute_tendencies: + value: a, b + type: array(string) + valid values: [] +$ ./atmchange homme::compute_tendencies+=c +$ ./atmquery homme::compute_tendencies --full + namelist_defaults::atmosphere_processes::homme::compute_tendencies + value: a, b, c + type: array(string) + valid values: [] +``` + +# Modifying the list of atmosphere processes + +The `atmchange` script can be used to change any of the runtime parameters of EAMxx. In particular, it can +be used to add, remove, or reorder atmosphere processes. When adding an atmosphere process, we must first +make sure that the defaults for that process are present in `namelist_defaults_scream.xml`. For instance, +the default settings for the "physics" atmosphere process group include the following: +```bash +$ ./atmquery physics::atm_procs_list + namelist_defaults::atmosphere_processes::physics::atm_procs_list: mac_aero_mic,rrtmgp +``` +where "mac_aero_mic" is itself an atmosphere process group, consisting of macrophysics, aerosols, and microphysics +processes. If one wanted to add the "cosp" atmosphere process to this list, and change the number of its +subcycles, it could do so via +```bash +$ ./atmchange physics::atm_procs_list+=cosp +$ ./atmchange cosp::number_of_subcycles=3 +``` +Notice that if we swapped the two commands, we would get an error, since the node "cosp" is not present in +the XML generated from the defaults until we decide to add it. + +It is also possible to declare a new (empty) atmosphere process group, which can then be filled with valid +atmosphere processes via subsequent calls to `atmchange`. The syntax to trigger this behavior consists +in specifying a process name that begins and ends with an underscore: +```bash +$ ./atmchange physics::atm_procs_list+=_my_group_ +``` +This adds a new process to the list of processes in "physics", called "\_my_group\_", and which is itself +an atmosphere process group. Hence, we can then do +```bash +$ ./atmchange _my_group_::atm_procs_list+=A,B +``` +where A and B must be valid atmosphere process names (i.e., present in `namelist_defaults_scream.xml`) +or be themselves new atmosphere process groups (i.e., beginning/ending with an underscore) + +`atmchange` can also be used to completely change a list of atmosphere processes: +```bash +$ ./atmchange physics::atm_procs_list=A,B,C +``` +Notice that we used "=" instead of "+=", which means we will be overwriting the value, rather than appending. +Any atmosphere process that was previously in the list but is no longer in it will be removed from +the generated `namelist_defaults.xml` (and `scream_input.yaml`) files, along with all their nested parameters. From def068302ef2f3b132d7b3cc7642d3e1cafeda22 Mon Sep 17 00:00:00 2001 From: Luca Bertagna Date: Tue, 30 Apr 2024 17:04:37 -0600 Subject: [PATCH 2/6] EAMxx: minor reworks of the user guide layout No need for the 'installation' section, which is only for standalone builds (and we never install scream anyways). Also, add a small md file where we can put some info about what compsets/grids we currently support for EAMxx --- components/eamxx/docs/user/eamxx_cases.md | 5 +++++ components/eamxx/docs/user/index.md | 6 ++++++ components/eamxx/mkdocs.yml | 6 ++---- 3 files changed, 13 insertions(+), 4 deletions(-) create mode 100644 components/eamxx/docs/user/eamxx_cases.md diff --git a/components/eamxx/docs/user/eamxx_cases.md b/components/eamxx/docs/user/eamxx_cases.md new file mode 100644 index 00000000000..2655d7782a7 --- /dev/null +++ b/components/eamxx/docs/user/eamxx_cases.md @@ -0,0 +1,5 @@ +Basics of EAMxx cases +===================================== + +This section explains how to create a case which uses EAMxx as the atmosphere model, +as well as what are the currently supported compsets and grids for EAMxx. diff --git a/components/eamxx/docs/user/index.md b/components/eamxx/docs/user/index.md index ba53083fc75..557c37c7aef 100644 --- a/components/eamxx/docs/user/index.md +++ b/components/eamxx/docs/user/index.md @@ -1,3 +1,9 @@ # SCREAM User Guide +This section contains documentation on how to create, setup, and run CIME cases with EAMxx as the atmosphere component. +It is assumed that the reader has a familiarity with [CIME case +control system](https://esmci.github.io/cime/versions/master/html/users_guide/index.html). In particular, we assume +that the user knows how to create a case, and what the `case.setup`, `case.build`, and `case.submit` commands do. + + For the time being, see our [public confluence EAMxx user guide](https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/3858890786/EAMxx+User+s+Guide) diff --git a/components/eamxx/mkdocs.yml b/components/eamxx/mkdocs.yml index ccf72c08039..a88b97ff0b9 100644 --- a/components/eamxx/mkdocs.yml +++ b/components/eamxx/mkdocs.yml @@ -4,10 +4,9 @@ nav: - 'Home': 'index.md' - 'User Guide': - 'Overview': 'user/index.md' - - 'Installation': 'common/installation.md' - - 'Model output': 'user/model_output.md' + - 'EAMxx case basics': 'user/eamxx_cases.md' - 'Model input': 'user/model_input.md' - - 'Runtime parameters': 'common/eamxx_params.md' + - 'Model output': 'user/model_output.md' - 'Nudging': 'user/nudging.md' - 'Extra radiation calls': 'user/clean_clear_sky.md' - 'Developer Guide': @@ -48,7 +47,6 @@ theme: features: - navigation.indices - navigation.instant - - navigation.sections - navigation.top # - navigation.tabs From 5561c5c38669a54f0e4581e8c024fbb9f20cbfb8 Mon Sep 17 00:00:00 2001 From: Luca Bertagna Date: Tue, 30 Apr 2024 17:06:35 -0600 Subject: [PATCH 3/6] EAMxx: avoid bad link error from mkdocs --- components/eamxx/docs/user/model_input.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/components/eamxx/docs/user/model_input.md b/components/eamxx/docs/user/model_input.md index 702bf7c89b7..5deb7c6dd3f 100644 --- a/components/eamxx/docs/user/model_input.md +++ b/components/eamxx/docs/user/model_input.md @@ -27,7 +27,7 @@ The main files/scripts involved in the EAMxx runtime configuration are the follo this XML file around at all. The main reason is to make it easier for the `atmquery` script (see below) to retrieve settings. 4. `namelist.nl`: EAMxx will parse this [namelist](https://docs.oracle.com/cd/E19957-01/805-4939/6j4m0vnad/index.html) - file at runtime, to retrieve all the parameters specific to the [Homme](some_homme_ref) dycore. + file at runtime, to retrieve all the parameters specific to the Homme (ADD REF) dycore. It is generated automatically by `buildnml` inside the RUNDIR/data folder, where `RUNDIR` is the run directory generated by CIME. Since this file is automatically generated when `buildnml` runs, users should not manually modify it. Any manual modification will be lost the next time `buildnml` runs From c5eaa89b178bb4f35c5d2221bf3dc3defbbc0c17 Mon Sep 17 00:00:00 2001 From: Luca Bertagna Date: Wed, 1 May 2024 11:59:15 -0600 Subject: [PATCH 4/6] EAMxx: improvements to the model input user guide docs --- components/eamxx/docs/user/model_input.md | 128 +++++++++++++--------- 1 file changed, 75 insertions(+), 53 deletions(-) diff --git a/components/eamxx/docs/user/model_input.md b/components/eamxx/docs/user/model_input.md index 5deb7c6dd3f..b0d662e5328 100644 --- a/components/eamxx/docs/user/model_input.md +++ b/components/eamxx/docs/user/model_input.md @@ -5,67 +5,66 @@ This section explains how input parameters are passed to EAMxx, and how the user change their value. The full list of the currently configuraable runtime parameters for EAMxx can be found [here](../common/eamxx_params.md). -The main files/scripts involved in the EAMxx runtime configuration are the following: - -1. `buildnml`: this script, located in `components/eamxx/cime_config`, is called by CIME's case management - scripts (`case.setup`, `case.build, `case.submit`), and is responsible for creating EAMxx input files, - including handling of user-specific modifications for the runtime parameters. Users should not have to - modify this script, nor should they have to manually call it. -2. `scream_input.yaml`: EAMxx will parse this [YAML](https://yaml.org/spec/1.2.2) file at runtime, - to detect all of its configuration parameters. It is generated automatically by `buildnml` inside the - RUNDIR/data folder, where `RUNDIR` is the run directory generated by CIME. Since this file is automatically - generated when `buildnml` runs, users should not manually modify it. Any manual modification will be - lost the next time `buildnml` runs (e.g., at `case.submit` time). -3. 'namelist_scream.xml`: this XML file is generated by `buildnml`, and contains all the runtime parameters that EAMxx - will read in at runtime. `buildnml` uses this XML file as an intermediate file during the generation of - `scream_input.yaml`. In particular, `buildnml` generates this file using case information to select the - proper configuration from the file `namelist_defaults_scream.xml`, located in `components/eamxx/cime_config`. - As it was the case for `scream_input.yaml`, since this XML file is automatically generated by `buildnml`, - the user should not manually modify this file. Any manual modification will be lost the next time `buildnml` - runs (e.g., at `case.submit` time). - Since the main product of `buildnml` is `scream_input.yaml`, one may wonder why we keep - this XML file around at all. The main reason is to make it easier for the `atmquery` script (see below) to - retrieve settings. -4. `namelist.nl`: EAMxx will parse this [namelist](https://docs.oracle.com/cd/E19957-01/805-4939/6j4m0vnad/index.html) - file at runtime, to retrieve all the parameters specific to the Homme (ADD REF) dycore. - It is generated automatically by `buildnml` inside the RUNDIR/data folder, where `RUNDIR` is the run - directory generated by CIME. Since this file is automatically generated when `buildnml` runs, - users should not manually modify it. Any manual modification will be lost the next time `buildnml` runs - (e.g., at `case.submit` time). -5. `atmchange` and `atmquery`: these script, located in `components/eamxx/scripts` and linked in the case folder, - can be used to query and change any of the runtime configuration parameters of EAMxx. - -Among the files listed above, `atmquery` and `atmchange` are the only ones that the user should interact with, -and therefore we give here a brief overview of how they can be used, and how their output can be interpreted. -For both these scripts, a short help can also be obtained using the `-h` flag, highlighting their use cases and options. +The infrastructure for reading inputs into EAMxx involves a few scripts/files: + +1. `atmchange` and `atmquery`: these scripts are located in `SRCDIR/components/eamxx/scripts`, + and are soft-linked in the case folder. As their names suggest, they can be used to query + and change the runtime configuration parameter of EAMxx. Since these two scripts are the + only scripts thta the average user needs to know and interact with, in the next sections + we give a brief overview of how they can be used and how their output can be interpreted. + Additionally, for both of the scripts a short help can also be obtained using the `-h` flag. +2. `buildnml`: this script, located in `SRCDIR/components/eamxx/cime_config`, is called by CIME's + case management scripts (`case.setup`, `case.build`, and `case.submit`), and is responsible for creating + the input files that EAMxx will read to load its runtime parameters. Users should not have to modify this script, + nor should they have to manually call it, but it is useful to know what it does. + When `buildnml` runs, it creates a few files, containing EAMxx input parameters: + - `scream_input.yaml`: this [YAML](https://yaml.org/spec/1.2.2) file is located in the `RUNDIR/data` folder, + and will be read by EAMxx at runtime to load all of its configuration parameters. More precisely, + this file contains parameters that need to be used inside the EAMxx interfaces. + - `namelist.nl`: this [namelist](https://docs.oracle.com/cd/E19957-01/805-4939/6j4m0vnad/index.html) file is + located in the `RUNDIR/data` folder, and will be parsed at runtime to get all the parameters for the + HOMME dycore (ADD REF). This file only contains dycore-specific parameters that are only recognized + inside HOMME, and does not contain any parameter pertaining EAMxx infrastructure. + - `namelist_scream.xml`: this XML file is located in the case directory, and contains all the runtime parameters that EAMxx + will read in at runtime. `buildnml` uses this XML file as an intermediate file during the generation of + `scream_input.yaml` and `namelist.nl`. More specifically, `buildnml` generates this file using case information to select the + proper configurations from the file `namelist_defaults_scream.xml`, located in `SRCDIR/components/eamxx/cime_config`. + Despite the fact that the only files that are needed at runtime are `scream_input.yaml` and `namelist.nl`, + we generate and keep this XML file around to make the implementation of `atmquery` easier. + + Since these files are automatically generated when `buildnml` runs, users should not manually modify them. + Any manual modification will be lost the next time `buildnml` runs (e.g., at `case.submit` time). # Querying parameters with `atmquery` This script is the simplest way for the user to check the value and properties of EAMxx input parameters. A basic usage of the script is -```bash +```shell $ ./atmquery my_param ``` + which will retrieve the value of the parameter called `my_param`, by locating the XML node "my_param" in the file `namelist_scream.xml` in the RUNDIR folder. Obviously, an XML file can have multiple nodes with the same tag, and the script is implemented to error out if multiple matches are found. In such a scenario, the user needs to provide also the parents nodes names, using enough parents to uniquely identify the node (in most cases, one parent is enough). To specify a parent, the user can prepend the parent name and `::` to the node name: -```bash +```shell $ ./atmquery foo::my_param ``` The output will contain the fully scoped parameter name, along with the value. E.g., -```bash + +```shell $ ./atmquery foo::my_param namelist_defaults::node1::node2::foo::my_param: 10 ``` It is sometimes desirable to query _all_ the nodes that have a particular name, or that contain a particular string. We can do that by using the `--grep` flag: -```blah + +```shell $ ./atmquery --grep sub iop_options::iop_dosubsidence: false ctl_nl::hypervis_subcycle: 1 @@ -76,11 +75,14 @@ $ ./atmquery --grep sub homme::number_of_subcycles: 1 physics::number_of_subcycles: 1 ``` -TODO: This difference between basica and `--grep` is not really intuitive (see [this -issue](https://github.com/E3SM-Project/scream/issues/2413)). + +TODO: This difference between basic and `--grep` is not really intuitive: as pointer out in [this +issue](https://github.com/E3SM-Project/scream/issues/2413), we should change this. If we do, don't forget +to update this following part of the docs. Using the `--grep` option has another effect: if the match is not a leaf of the XML tree, all its subelements are printed: -```bash + +```shell $ ./atmquery --grep homme homme Moisture: moist @@ -94,9 +96,10 @@ $ ./atmquery --grep homme ``` Similarly to the CIME utility `xmlchange`, the options `--value`, `--type`, `--valid-values`, and `--full` can be -used to respectively retrieve just the parameter value (useful for bash scripting), the parameter's type, +used to respectively retrieve just the parameter value (useful for shell scripting), the parameter's type, a list of valid values for parameter (when applicable), or all of the above: -```bash + +```shell $ ./atmquery atm_log_level --value info $ ./atmquery atm_log_level --type @@ -112,7 +115,8 @@ $ ./atmquery atm_log_level --full Finally, the option `--listall` can be used to list the whole content of the XML file, which will be displayed with each node indented in its parent scope: -```bash + +```shell $ ./atmquery --listall namelist_defaults grids_manager @@ -134,12 +138,14 @@ the compset, etc.) and the `namelist_scream_defaults.xml` file, located in the e The user can change any of these parameters using the `atmchange` script. A basic usage of the script is -```bash +```shell $ ./atmchange my_param=10 ``` + As for `atmquery`, if there are multiple matches for a given parameter name, the user must specify a unique scoped name, which allows `atmchange` to uniquely identify the XML node to modify: -```bash + +```shell $ ./atmquery homme::number_of_subcycles namelist_defaults::atmosphere_processes::homme::number_of_subcycles: 1 $ ./atmchange number_of_subcycles=10 @@ -160,9 +166,11 @@ Regenerating /path/to/namelist_scream.xml. Manual edits will be lost. $ ./atmquery homme::number_of_subcycles namelist_defaults::atmosphere_processes::homme::number_of_subcycles: 10 ``` + In some cases, the user may be interested in changing _all_ nodes with a given name. In that case, the `--all` flag can be used: -```bash + +```shell $ ./atmquery --grep number_of_subcycles atmosphere_processes::number_of_subcycles: 1 sc_import::number_of_subcycles: 1 @@ -192,14 +200,17 @@ $ ./atmquery --grep number_of_subcycles rrtmgp::number_of_subcycles: 3 sc_export::number_of_subcycles: 3 ``` + Since the XML file stores constraints on the parameter value (like its type or valid values), attempting to use the wrong type will cause an error: -```bash + +```shell $ ./atmquery --type se_ne namelist_defaults::ctl_nl::se_ne: integer $ ./atmchange se_ne=hello ERROR: Could not refine 'hello' as type 'integer': ``` + There are three main types supported: integer, float, string, logical. When passing a string to `atmchange`, the script will try to interpret it acoording to the parameter type, and throw an error if that's not possible: for "string", anything works; for "integer", only digits are allowed, possibly with a negative sign in front; @@ -212,7 +223,8 @@ or certain lookup tables. a `std::vector`. For type "string" and "array(T)", it is also possible to _append_ to the currently stored value -```bash + +```shell $ ./atmquery homme::compute_tendencies namelist_defaults::atmosphere_processes::homme::compute_tendencies: value: a, b @@ -232,38 +244,48 @@ The `atmchange` script can be used to change any of the runtime parameters of EA be used to add, remove, or reorder atmosphere processes. When adding an atmosphere process, we must first make sure that the defaults for that process are present in `namelist_defaults_scream.xml`. For instance, the default settings for the "physics" atmosphere process group include the following: -```bash + +```shell $ ./atmquery physics::atm_procs_list namelist_defaults::atmosphere_processes::physics::atm_procs_list: mac_aero_mic,rrtmgp ``` + where "mac_aero_mic" is itself an atmosphere process group, consisting of macrophysics, aerosols, and microphysics processes. If one wanted to add the "cosp" atmosphere process to this list, and change the number of its subcycles, it could do so via -```bash + +```shell $ ./atmchange physics::atm_procs_list+=cosp $ ./atmchange cosp::number_of_subcycles=3 ``` + Notice that if we swapped the two commands, we would get an error, since the node "cosp" is not present in the XML generated from the defaults until we decide to add it. It is also possible to declare a new (empty) atmosphere process group, which can then be filled with valid atmosphere processes via subsequent calls to `atmchange`. The syntax to trigger this behavior consists in specifying a process name that begins and ends with an underscore: -```bash + +```shell $ ./atmchange physics::atm_procs_list+=_my_group_ ``` + This adds a new process to the list of processes in "physics", called "\_my_group\_", and which is itself an atmosphere process group. Hence, we can then do -```bash + +```shell $ ./atmchange _my_group_::atm_procs_list+=A,B ``` + where A and B must be valid atmosphere process names (i.e., present in `namelist_defaults_scream.xml`) or be themselves new atmosphere process groups (i.e., beginning/ending with an underscore) `atmchange` can also be used to completely change a list of atmosphere processes: -```bash + +```shell $ ./atmchange physics::atm_procs_list=A,B,C ``` + Notice that we used "=" instead of "+=", which means we will be overwriting the value, rather than appending. Any atmosphere process that was previously in the list but is no longer in it will be removed from the generated `namelist_defaults.xml` (and `scream_input.yaml`) files, along with all their nested parameters. From 09ce61e0c5ba87d328fde8a27fcf43087341f33d Mon Sep 17 00:00:00 2001 From: Luca Bertagna Date: Wed, 1 May 2024 15:00:42 -0600 Subject: [PATCH 5/6] EAMxx: fix typos in docs --- components/eamxx/docs/user/model_input.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/components/eamxx/docs/user/model_input.md b/components/eamxx/docs/user/model_input.md index b0d662e5328..fb25422846e 100644 --- a/components/eamxx/docs/user/model_input.md +++ b/components/eamxx/docs/user/model_input.md @@ -10,7 +10,7 @@ The infrastructure for reading inputs into EAMxx involves a few scripts/files: 1. `atmchange` and `atmquery`: these scripts are located in `SRCDIR/components/eamxx/scripts`, and are soft-linked in the case folder. As their names suggest, they can be used to query and change the runtime configuration parameter of EAMxx. Since these two scripts are the - only scripts thta the average user needs to know and interact with, in the next sections + only scripts that the average user needs to know and interact with, in the next sections we give a brief overview of how they can be used and how their output can be interpreted. Additionally, for both of the scripts a short help can also be obtained using the `-h` flag. 2. `buildnml`: this script, located in `SRCDIR/components/eamxx/cime_config`, is called by CIME's @@ -76,7 +76,7 @@ $ ./atmquery --grep sub physics::number_of_subcycles: 1 ``` -TODO: This difference between basic and `--grep` is not really intuitive: as pointer out in [this +TODO: This difference between basic and `--grep` is not really intuitive: as pointed out in [this issue](https://github.com/E3SM-Project/scream/issues/2413), we should change this. If we do, don't forget to update this following part of the docs. Using the `--grep` option has another effect: if the match is not a leaf of the XML tree, all its subelements From 4600445a8045a10c2cc7dbe31162bc203bbc818d Mon Sep 17 00:00:00 2001 From: Luca Bertagna Date: Thu, 2 May 2024 09:54:29 -0600 Subject: [PATCH 6/6] EAMxx: docs rephrase pointer to confluence docs --- components/eamxx/docs/user/index.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/components/eamxx/docs/user/index.md b/components/eamxx/docs/user/index.md index 557c37c7aef..2e24e9fa736 100644 --- a/components/eamxx/docs/user/index.md +++ b/components/eamxx/docs/user/index.md @@ -6,4 +6,5 @@ control system](https://esmci.github.io/cime/versions/master/html/users_guide/in that the user knows how to create a case, and what the `case.setup`, `case.build`, and `case.submit` commands do. -For the time being, see our [public confluence EAMxx user guide](https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/3858890786/EAMxx+User+s+Guide) +This user guide is still under construction. In the meantime, in case you can't find the information you need, +you may visit our public confluence [EAMxx user guide](https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/3858890786/EAMxx+User+s+Guide).