Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update conventional station pressure (conv_ps) for testing and end-to-end #698

Merged
merged 17 commits into from
Nov 2, 2023

Conversation

emilyhcliu
Copy link
Collaborator

@emilyhcliu emilyhcliu commented Oct 31, 2023

This PR is to update the station pressure with the following modifications:

  • Modify YAMLs under testing (UFO Evaluation)

    • Add error assignment by extracting errors from the GSI error table (needs an update from UFO PR #3085)
    • Update options for error inflation from pressure check (due to updates from GMAO)
  • Add YAMLs under config (End-to-End)
    This will be tested during the code sprint.

  • Add handling for CI tests with and without QC

To closely replicate the error assignment from the GSI error table. The ObsFunctionModelStepwiseLinear needs to add the capability of rounding the output of this function to the nearest integer. A related https://github.com/JCSDA-internal/ufo/pull/3085 has been submitted in the UFO repository.

Notes for test data set:

  • The error inflation for duplicate observation is turned on for the UFO evaluation.
  • The input obs file is a combined data type from 180 183 (SFCSHIP), 181, 187 (SFC), and 120 (Radiosonde Ps) for stationPressure.
  • The GSI input obs error read from the prepbufr file is stored in the obs file for reference and validation. This can be used to validate the obs error values extracted from the errtable using ObsErrorModelStepwiseLinear function.

Please see T2O Issue #96 for documentation and UFO evaluation test results.

The CI test for conv_ps should change to green after merging this PR.

…o error inflation for duplicate observations and the JEDI window length definition, the window length in the script has to be relaxed so that all observations from the 6-hour obs data will be accounted for.
@emilyhcliu emilyhcliu changed the title Update conventional station pressure (conv_ps) for testing Update conventional station pressure (conv_ps) for testing and end-to-end Oct 31, 2023
CoryMartin-NOAA
CoryMartin-NOAA previously approved these changes Nov 1, 2023
@CoryMartin-NOAA
Copy link
Contributor

CoryMartin-NOAA commented Nov 1, 2023

@emilyhcliu can you resolve the conflicts and merge in develop, then I think this is ready to go! (well once the JCSDA merges the UFO pr)

@emilyhcliu
Copy link
Collaborator Author

@emilyhcliu can you resolve the conflicts and merge in develop, then I think this is ready to go! (well once the JCSDA merges the UFO pr)

@CoryMartin-NOAA Done: Updated the branch, resolved the conflict, and did git push.

CoryMartin-NOAA
CoryMartin-NOAA previously approved these changes Nov 1, 2023
Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, will not merge until UFO PR is merged

Copy link
Collaborator

@PraveenKumar-NOAA PraveenKumar-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, except one general question: why surface_geometric_height is replaced by surface_altitude for geovar_sfc_geomz?

@PraveenKumar-NOAA Thanks for reviewing this PR.
You can use either one.
The surface_altitude is the default and then GMAO added a second default (surface_geometric_height)

Copy link
Collaborator

@PraveenKumar-NOAA PraveenKumar-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@emilyhcliu
Copy link
Collaborator Author

@CoryMartin-NOAA I updated the test_yamls.sh to run CI test with and without QC.

Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @emilyhcliu!

@emilyhcliu
Copy link
Collaborator Author

@CoryMartin-NOAA The UFO PR related to this PR has been approved and merged into UFO. We are good to merge if Mr. Goofy (@ADCollard ) agrees with the changes in test_yaml.sh.

@CoryMartin-NOAA
Copy link
Contributor

image

@CoryMartin-NOAA CoryMartin-NOAA merged commit 19701ac into develop Nov 2, 2023
5 checks passed
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/conv_ps branch November 2, 2023 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants