Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recent WRF-Hydro Developments #638

Merged
merged 7 commits into from
Mar 27, 2024
Merged

Conversation

mgharamti
Copy link
Contributor

@mgharamti mgharamti commented Feb 15, 2024

Description:

Changes:

  • Allow pmo in WRF-Hydro
  • Enhance the performance for big domains, e.g., CONUS
  • Faster Localization
  • Enhanced diagnostics
  • Better handling of streamflow identity obs

Fixes issue

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Documentation changes needed?

  • My change requires a change to the documentation.
    • I have updated the documentation accordingly.

Tests

Please describe any tests you ran to verify your changes.

Checklist for merging

  • Updated changelog entry
  • Documentation updated
  • Update conf.py

Checklist for release

  • Merge into main
  • Create release from the main branch with appropriate tag
  • Delete feature-branch

Testing Datasets

  • Dataset needed for testing available upon request
  • Dataset download instructions included
  • No dataset needed

A new perfect model obs experimental capability is added
to HydroDART. The python and shell scripts use a user-defined
truth run to sample psuedo-observations and run OSSEs.
The following scripts have been tested in regional hydroDART
runs such as the DRB domain.

1- gen_truth.sh:
Starting from a list of wrf-hydro restarts (truth), strip out streamflow
information at all available routelink gauges or at a subset of gauges specified
by the user. The resulting netcdf files are stored in a new directory.

2- gauges.sh:
A script to handle the gauges in the domain. It can be used to identify a subset
"desired" set of gauges or retrive all gauges available in wrf_hydro's
Routelink file.

3- pmo_osse.py:
This iterates over the truth files obtained by running 'gen_truth.sh'
and builds a set of obs sequence files. These are then used by the
hydroDART famework to conduct an OSSE. Credits to Ben Johnson who
built initial versions of this script.

PS. This could have been done by building a wrapper around DART's
own perfect_model_obs routine. The reasons why we went this route
are unknown at this time.
Streamflow obs converter is modified to allow for better diagnostics.
The changes allows DART to compute obs space diagnostics on all gauges
from the Routelink (not only the ones specified by the user). This
essentially mimcs evaluate_only functionality for non-idenity obs.
There is also a change to check for bad (inf/nan) streamflow obs.
A final change allows processing a large number of obs-seq files
at the same time.

The python scripts include bug fixes for file movement. It also
allows to call the converter from the yaml file using fat memory nodes.
Changes to the model_mod and noah_hydro_mod to enhance performance
when running a full CONUS domain. Various parts of the noah_hydro_mod
code are rearranged to avoid extra computations that are not needed
when the LSM is turned off. Also, the link tree structure is modified
such that only the individual upstream links from any reach are stored.
We also limit the number of upstream links to '5'. This helps alleviate
any issues in the Routelink file (which can make the stream network unphysical).
This change makes our Along-The-Stream (ATS) Localization runs a lot faster.

Changes in the model_mod makes the code faster when distributing the close
by reaches to each individual task. Also, there is a change that allows reading
climatology files when a hybrid EnKF variant is invoked.
Modifications to hydroDART's diagnostic routine now allows the following:
1. Save the hydrographs in a high-resolution pdf
2. Handles hybrid DA components (weight mean and sd)
3. Better parsing of the netcdf files
4. Allows the openloop to have different ens size (compared to the regular DA runs)
5. Better time handling
6. Collects more information about the openloop run
7. The openloop may have different gauges than the DA runs
8. Better usage of matlab functions (removing obselete ones)
9. Separate plots for the hybrid statistics
@hkershaw-brown
Copy link
Member

Hi Moha, checking in with what you're looking for in this review. Anything in particular you want reviewers to take a close look at?

Cheers,
Helen

@mgharamti
Copy link
Contributor Author

@hkershaw-brown I would like mainly the 'enhanced performance' commit to be looked at. The other commits are less important and have been tested extensively. The enhanced performance commit has changes to model-mod and noah-hydro-mod to accelerate several computations. Please let me know if you have any questions.

Copy link
Member

@hkershaw-brown hkershaw-brown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Moha,

First look though this, there are quite a few lines of code commented out. It would be good to remove the commented out code if it is no longer being used.

Cheers,
Helen

Comment on lines 1047 to 1052
!io = nf90_inq_varid(ncid,'gages', VarID)
!call nc_check(io, routine, 'inq_varid', 'gages', filename)
!io = nf90_get_var(ncid, VarID, gageID)
!call nc_check(io, routine, 'get_var', 'gages', filename)

!call nc_close_file(ncid, routine)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are quite a few commented out lines of code. It would be better to remove the code if it is not being used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Member

@hkershaw-brown hkershaw-brown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved!

@hkershaw-brown hkershaw-brown added the release! bundle with next release label Mar 21, 2024
@hkershaw-brown hkershaw-brown mentioned this pull request Mar 22, 2024
15 tasks
@mjs2369 mjs2369 changed the base branch from main to airs-converter-docs March 27, 2024 23:00
@mjs2369 mjs2369 merged commit 95c0db6 into NCAR:airs-converter-docs Mar 27, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release! bundle with next release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants