Bugfix for METviewer Scorecard

[michelleharrold]

Replace italics below with details for this issue.

Describe the Problem

@briannen and I are working on applying interactive scorecards to MET output from the DTC RRFS Agile Framework Team. We are able to generate correct scorecards from older MET output (v10.1.1), but with more current (v11.1.0) MET output the scorecard does not generate correct output. A scorecard is created, but all cells are 'Not statistically relevant.'

Expected Behavior

Using a similar (but not the same, since the underlying MET output is different) scorecard XML, I would expect a correct scorecard with statistically relevant data for all cells with both METv10.1.1 and METv11.1.0 data.

Environment

Describe your runtime environment:
Testing is taking place on Dakota. The scorecard script we are pointing to is: /d3/projects/METViewer/src_scorecard/apps/METviewer/bin/mv_scorecard.sh

A test XML using METv10.1.1 data is here:
/d3/projects/RRFS/metv/interactive/scorecard_rrfs.xml
(output: /d3/projects/RRFS/metv/interactive/scorecard_rrfs_sfc.png)

A test XML using METv11.1.0 data is here:
/d3/projects/RRFS/metv/interactive/briannen/scorecard_test.xml
(log: /d3/projects/RRFS/metv/interactive/briannen/test.log)
(output: /d3/projects/RRFS/metv/interactive/briannen/scorecard_hwt_2mT.png)

To Reproduce

Describe the steps to reproduce the behavior:
On dakota:
/d3/projects/METViewer/src_scorecard/apps/METviewer/bin/mv_scorecard.sh /d3/projects/RRFS/metv/interactive/briannen/scorecard_test.xml

Relevant Deadlines

Ideally, we would like a fix by mid-November, but if not, we would like to have this fixed by Jan. for the Annual AMS meeting.

Funding Source

FY23 NOAA Base (2792533) [NOTE: Subject to change to NOAA FY24!]

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects

• Select Milestone as the next bugfix version
• Select Coordinated METplus-X.Y Support project for support of the current coordinated release
• Select METviewer-X.Y.Z Development project for development toward the next official release

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Bugfix Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of main_<Version>.
  Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
• Fix the bug and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into main_<Version>.
  Pull request: bugfix <Issue Number> main_<Version> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next bugfix version
  Select: Coordinated METplus-X.Y Support project for support of the current coordinated release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Complete the steps above to fix the bug on the develop branch.
  Branch name: bugfix_<Issue Number>_develop_<Description>
  Pull request: bugfix <Issue Number> develop <Description>
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: METviewer-X.Y.Z Development project for development toward the next official release
• Close this issue.

[bikegeek]

Log message from the scorecard using METv11.1 data indicates there is an error:
2024-09-17T16:26:16,221 ERROR [-1] [addJsonRow] e.u.m.s.GraphicalOutputManager:
WARNING: one of the fields contains '-'. The scorecard could be invalid!!!!!

This is traced to the edu.ucar.metviewer.scorecard.GraphicalOutputManager.java addJsonRow method. The model name is checked for the presence of more than one '-' and if that condition is met, raises a warning and then replaces '-' with whitespace.

This has been verified by a database query for data from METv10.x (mv_rrfs_href_spring2022) and METv11.1 (mv_hwt2024):

MET10.x data:

MariaDB [mv_rrfs_href_spring2022]> select distinct(model) from stat_header;
+------------+
| model |
+------------+
| HREF |
| HREF_mean |
| HREF_mem01 |
| HREF_mem02 |
| HREF_mem03 |
| HREF_mem04 |
| HREF_mem05 |
| HREF_mem06 |
| HREF_mem07 |
| HREF_mem08 |
| HREF_mem09 |
| HREF_mem10 |
| HREF_prob |
| RRFS |
| RRFS_mean |
| RRFS_mem01 |
| RRFS_mem02 |
| RRFS_mem03 |
| RRFS_mem04 |
| RRFS_mem05 |
| RRFS_mem06 |
| RRFS_mem07 |
| RRFS_mem08 |
| RRFS_mem09 |
| RRFS_mem10 |
| RRFS_prob |
+------------+

MET 11.1 data:

MariaDB [mv_hwt2024]> select distinct model from stat_header;
+---------------------+
| model |
+---------------------+
| GSL-MPAS05_mem000 |
| HRRR_mem000 |
| NSSL-MPAS-HN_mem000 |
| NSSL-MPAS-HT_mem000 |
| NSSL-MPAS-RT_mem000 |
+---------------------+

The METv11.1 has model names with more than one '-' present. This is most likely causing inadvertent "mangling" of the DIFF_SIG model name in METv11.1 data (and most likely affecting the generation of correct/expected statistical results):

Note the absence of the minus sign between the two models for the METv11.1 data:
DIFF_SIG(Z2 HRRR_mem000 350000 TMP ME-Z2 NSSL-MPAS-HN_mem000 350000 TMP ME)

Note the '-' to indicate the minus sign in the METv10.x data (which is expected/correct):

METv10.x:
DIFF_SIG(Z10 HREF_mem04 360000 WIND BCRMSE-Z10 RRFS_mem01 360000 WIND BCRMSE)

[michelleharrold]

@bikegeek -- wooot! Thanks for tracking that down so quickly. Going forward, would it be possible to have more than one '-' present, or should we change our model name? If it is possible to only have one '-' present, we might want to document that, so users are aware.

Thank you!

[bikegeek]

I think moving forward we should allow model names with more than one '-'.

…

On Fri, Oct 25, 2024 at 4:12 PM michelleharrold ***@***.***> wrote: @bikegeek <https://github.com/bikegeek> -- wooot! Thanks for tracking that down so quickly. Going forward, would it be possible to have more than one '-' present, or should we change our model name? If it is possible to only have one '-' present, we might want to document that, so users are aware. Thank you! — Reply to this email directly, view it on GitHub <#553 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4UJHRD2ASSXHHSLWTW2XDZ5K64FAVCNFSM6AAAAABQO5R366VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZYHE2TAOBRGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bikegeek]

The Error raised from the GraphicalOutputManager might not be the cause of the error, it might just be the check that indicates something is wrong. I've set up an IDE on seneca to step through the code to see where the DIFF_STAT statistics are being computed.

[michelleharrold]

thanks @bikegeek -- if you need someone to help test, work through this, please let me or @briannen know!

[michelleharrold]

After reloading a database that changed the model names with multiple dashes to have underscores instead, I was able to make a scorecard successfully! So this error does seem to be directly related to multiple dashes.