Global-det SST sea ice fix #660

QiShi-NOAA · 2025-01-31T20:16:35Z

Note to developers: You must use this PR template!

Description of Changes

Please include a summary of the changes and the related GitHub issue(s). Please also include relevant motivation and context.

This PR fixes the SST issue with ice temp definition discrepancies between the forecasts and obs for global_det. In GridStat_fcstGLOBAL_DET_obsGHRSST_OSPO_DailyAvg.conf, "cnt_thresh = [ >= 271.15 ]; cnt_logic = INTERSECTION;" was set under FCST_VAR1_OPTIONS and OBS_VAR1_OPTIONS. Fixed plotting issue of fixed SST stat files by making changes in global_det_atmos_plots_specs.py.

Developer Questions and Checklist

Is this a high priority PR? If so, why and is there a date it needs to be merged by?
Yes, this PR is needed for official validation.
Do you have any planned upcoming annual leave/PTO? No
Are there any changes needed in the times when the jobs are supposed to run/kick-off? No

The code changes follow NCO's EE2 Standards.
Developer's name is removed throughout the code and have used ${USER} where necessary throughout the code.
References the feature branch for HOMEevs are removed from the code.
J-Job environment variables, COMIN and COMOUT directories, and output follow what has been defined for EVS.
Jobs over 15 minutes in runtime have restart capability.
If applicable, changes in the dev/drivers/scripts or dev/modulefiles have been made in the corresponding ecf/scripts and ecf/defs/evs-nco.def?
Jobs contain the appropriate file checking and don't run METplus for any missing data.
Code is using METplus wrappers structure and not calling MET executables directly.
Log is free of any ERRORs or WARNINGs.

Testing Instructions

Please include testing instructions for the PR assignee. Include all relevant input datasets needed to run the tests.

Checkout this feature branch and set up FIXevs and COMOUT accordingly.
Test the SST G2G stats jobs by running the following:
dev/drivers/scripts/stats/global_det/jevs_global_det_gfs_atmos_grid2grid_stats.sh
After confirming the successful stats jobs, test the SST plot jobs by running:
dev/drivers/scripts/plots/global_det/jevs_global_det_atmos_grid2grid_sst_plots_last31days.sh
jevs_global_det_atmos_grid2grid_sst_plots_last90days.sh

* Edit stats and plots scripts for SST ice mask fix * Remove threshold value from SST plot title and file name

malloryprow · 2025-02-03T13:22:22Z

We had operational failures in operations (and in the parallel too) due to MET not being able to read the GHRSST-OSPO file. I'm going to work with the MET team to see what is wrong with the file. Until we see what is going on it will be hard to test this PR.

QiShi-NOAA · 2025-02-03T13:27:45Z

Hi Mallory, Thanks for the information. I'll work on the wind direction while I'm waiting. Best, Qi

…

On Mon, Feb 3, 2025 at 8:22 AM Mallory Row ***@***.***> wrote: We had operational failures in operations (and in the parallel too) due to MET not being able to read the GHRSST-OSPO file. I'm going to work with the MET team to see what is wrong with the file. Until we see what is going on it will be hard to test this PR. — Reply to this email directly, view it on GitHub <#660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BMPTN22XXN4H5P5GQY7DOGL2N5USHAVCNFSM6AAAAABWIKPSNKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZQHE4TIOJZGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

malloryprow · 2025-02-05T16:35:21Z

Can you do a sync fork? Once you do that I can test this now that the fix for reading the GHRSST_OSPO data is in develop.

QiShi-NOAA · 2025-02-05T16:37:40Z

Hi Mallory, I've sync the fork. Best, Qi

…

On Wed, Feb 5, 2025 at 11:35 AM Mallory Row ***@***.***> wrote: Can you do a sync fork? Once you do that I can test this now that the fix for reading the GHRSST_OSPO data is in develop. — Reply to this email directly, view it on GitHub <#660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BMPTN27GWTAPYYWX2U67V732OI4WBAVCNFSM6AAAAABWIKPSNKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZXGQZTSNJWGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

malloryprow · 2025-02-05T17:50:37Z

1. jevs_global_det_gfs_atmos_grid2grid_stats.sh

COMOUT: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/evs/v2.0/stats/global_det
Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/EVS/dev/drivers/scripts/stats/global_det/jevs_global_det_gfs_atmos_grid2grid_stats_00.o6421990
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_global_det_gfs_atmos_grid2grid_stats_00.6421990.dbqs01

QiShi-NOAA · 2025-02-05T18:17:35Z

1. jevs_global_det_gfs_atmos_grid2grid_stats.sh

COMOUT: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/evs/v2.0/stats/global_det Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/EVS/dev/drivers/scripts/stats/global_det/jevs_global_det_gfs_atmos_grid2grid_stats_00.o6421990 DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_global_det_gfs_atmos_grid2grid_stats_00.6421990.dbqs01

The ooutput files and log file look good. No error is found.

malloryprow · 2025-02-05T18:19:23Z

For testing the SST plots, do you have an archive I can use to test with the new stat files?

QiShi-NOAA · 2025-02-05T18:30:33Z

I don't have an archive, I've tested my plot on two day stats: /lfs/h2/emc/vpppg/noscrub/qi.shi/EVS_dev_qi/evs/v2.0/stats/global_det Let me know if you want me to generate more stats.

…

On Wed, Feb 5, 2025 at 1:19 PM Mallory Row ***@***.***> wrote: For testing the SST plots, do you have an archive I can use to test with the new stat files? — Reply to this email directly, view it on GitHub <#660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BMPTN22DDBE5ZGEZI6TO5332OJI4FAVCNFSM6AAAAABWIKPSNKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZXGY4DSNZZGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

malloryprow · 2025-02-07T13:07:07Z

1. jevs_global_det_atmos_grid2grid_sst_plots_last31days

COMOUT: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/evs/v2.0/plots/global_det/atmos.20250206
Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/EVS/dev/drivers/scripts/plots/global_det/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.o6595491
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.6595491.dbqs01

My plots are blank.

I believe this PR is missing threshold information in ush/global_det/global_det_atmos_plots_grid2grid_create_job_scripts.py. See the changes in #657.

…pts.py

QiShi-NOAA · 2025-02-07T14:01:31Z

I've added the SST threshold to the ush/global_det/global_det_atmos_plots_grid2grid_create_job_scripts.py.

…

On Fri, Feb 7, 2025 at 8:07 AM Mallory Row ***@***.***> wrote: 1. jevs_global_det_atmos_grid2grid_sst_plots_last31days *COMOUT:* /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/evs/v2.0/plots/global_det/atmos.20250206 *Log File:* /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/EVS/dev/drivers/scripts/plots/global_det/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.o6595491 *DATA:* /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.6595491.dbqs01 My plots are blank. Screenshot.from.2025-02-07.08-05-03.png (view on web) <https://github.com/user-attachments/assets/aebc18cb-ab3e-4d62-9b42-7bd282376b41> I believe this PR is missing threshold information in ush/global_det/global_det_atmos_plots_grid2grid_create_job_scripts.py. See the changes in #657 <#657>. — Reply to this email directly, view it on GitHub <#660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BMPTN26WLZEABOKVEFUXDVD2OSVZFAVCNFSM6AAAAABWIKPSNKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBSHA3DCNBUG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

malloryprow · 2025-02-07T14:14:35Z

1. jevs_global_det_atmos_grid2grid_sst_plots_last31days

COMOUT: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/evs/v2.0/plots/global_det/atmos.20250206
Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/EVS/dev/drivers/scripts/plots/global_det/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.o6605045
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.6605045.dbqs01

Looks much better now!

QiShi-NOAA · 2025-02-07T14:33:25Z

Both the log file and outputs look good. No error found.

…

On Fri, Feb 7, 2025 at 9:15 AM Mallory Row ***@***.***> wrote: 1. jevs_global_det_atmos_grid2grid_sst_plots_last31days *COMOUT:* /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/evs/v2.0/plots/global_det/atmos.20250206 *Log File:* /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr660/EVS/dev/drivers/scripts/plots/global_det/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.o6605045 *DATA:* /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_global_det_atmos_grid2grid_sst_plots_last31days_00.6605045.dbqs01 Looks much better now! Screenshot.from.2025-02-07.09-12-17.png (view on web) <https://github.com/user-attachments/assets/57352ec8-dba8-47ab-a078-58223b0f8b5c> — Reply to this email directly, view it on GitHub <#660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BMPTN2Z3EADDKWVR3LC7UJT2OS5WHAVCNFSM6AAAAABWIKPSNKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBTGA2TINBQGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

malloryprow

Code changes are good and testing successful.

Thanks are these changes @QiShi-NOAA! Check of the corresponding item for this in the Fixes and Additions document.

AliciaBentley-NOAA

I have reviewed this PR and found all of the changes made to be consistent with the SST fix that Mallory and Lydia described. I approve this PR to be merged.

CC @malloryprow @QiShi-NOAA

malloryprow · 2025-02-07T15:06:44Z

The parallel code has incorporated these changes, so when the global_det stats run later today (for VDATE=20250206) they will have the updated SST stats.

QiShi-NOAA · 2025-02-07T15:14:11Z

Thanks for the information. I'll be monitoring the SST plots.

…

On Fri, Feb 7, 2025 at 10:07 AM Mallory Row ***@***.***> wrote: The parallel code has incorporated these changes, so when the global_det stats run later today (for VDATE=20250206) they will have the updated SST stats. — Reply to this email directly, view it on GitHub <#660 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BMPTN25AWMHNJYZKJXVGCQL2OTDZRAVCNFSM6AAAAABWIKPSNKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBTGE4TQNRQHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Qi Shi and others added 2 commits January 31, 2025 19:44

Feature/global-det sst sea ice fix

1744171

* Edit stats and plots scripts for SST ice mask fix * Remove threshold value from SST plot title and file name

Merge branch 'NOAA-EMC:develop' into SST_sea_ice

316156a

malloryprow requested review from malloryprow and AliciaBentley-NOAA January 31, 2025 20:23

malloryprow self-assigned this Jan 31, 2025

malloryprow added the enhancement New feature or request label Jan 31, 2025

malloryprow added this to the EVS v2.0.0 milestone Jan 31, 2025

Merge branch 'NOAA-EMC:develop' into SST_sea_ice

71a93b2

Add SST threshold to global_det_atmos_plots_grid2grid_create_job_scri…

e44cace

…pts.py

malloryprow approved these changes Feb 7, 2025

View reviewed changes

AliciaBentley-NOAA approved these changes Feb 7, 2025

View reviewed changes

malloryprow merged commit 34e65b3 into NOAA-EMC:develop Feb 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Global-det SST sea ice fix #660

Global-det SST sea ice fix #660

QiShi-NOAA commented Jan 31, 2025

malloryprow commented Feb 3, 2025

QiShi-NOAA commented Feb 3, 2025 via email

malloryprow commented Feb 5, 2025

QiShi-NOAA commented Feb 5, 2025 via email

malloryprow commented Feb 5, 2025

QiShi-NOAA commented Feb 5, 2025

1. jevs_global_det_gfs_atmos_grid2grid_stats.sh

malloryprow commented Feb 5, 2025

QiShi-NOAA commented Feb 5, 2025 via email

malloryprow commented Feb 7, 2025

QiShi-NOAA commented Feb 7, 2025 via email

malloryprow commented Feb 7, 2025

QiShi-NOAA commented Feb 7, 2025 via email

malloryprow left a comment

AliciaBentley-NOAA left a comment

malloryprow commented Feb 7, 2025

QiShi-NOAA commented Feb 7, 2025 via email

Global-det SST sea ice fix #660

Global-det SST sea ice fix #660

Conversation

QiShi-NOAA commented Jan 31, 2025

Description of Changes

Developer Questions and Checklist

Testing Instructions

malloryprow commented Feb 3, 2025

QiShi-NOAA commented Feb 3, 2025 via email

malloryprow commented Feb 5, 2025

QiShi-NOAA commented Feb 5, 2025 via email

malloryprow commented Feb 5, 2025

1. jevs_global_det_gfs_atmos_grid2grid_stats.sh

QiShi-NOAA commented Feb 5, 2025

1. jevs_global_det_gfs_atmos_grid2grid_stats.sh

malloryprow commented Feb 5, 2025

QiShi-NOAA commented Feb 5, 2025 via email

malloryprow commented Feb 7, 2025

1. jevs_global_det_atmos_grid2grid_sst_plots_last31days

QiShi-NOAA commented Feb 7, 2025 via email

malloryprow commented Feb 7, 2025

1. jevs_global_det_atmos_grid2grid_sst_plots_last31days

QiShi-NOAA commented Feb 7, 2025 via email

malloryprow left a comment

Choose a reason for hiding this comment

AliciaBentley-NOAA left a comment

Choose a reason for hiding this comment

malloryprow commented Feb 7, 2025

QiShi-NOAA commented Feb 7, 2025 via email