Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update JGDAS_ENKF_SFC job with COMIN/COMOUT prefix instead of COM #3291

Open
wants to merge 10 commits into
base: develop
Choose a base branch
from
Open

Update JGDAS_ENKF_SFC job with COMIN/COMOUT prefix instead of COM #3291

wants to merge 10 commits into from

Conversation

mingshichen-noaa
Copy link
Contributor

Description

NCO has requested that each COM variable specify whether it is an input or an output. This completes that process for the global JGDAS_ENKF_SFC and JGDAS_ENKF_ARCHIVE jobs.

Refs #2451

Type of change

  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO

How has this been tested?

  • Clone and build on RDHPCS
  • Cycled tests on Hercules
  • Forecast-only tests on Hercules

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

@KateFriedman-NOAA
Copy link
Member

@mingshichen-noaa The prior incarnation of this PR made COM changes for the enkf_sfc and the enkf_update job but this new PR includes COM changes for the enkf_sfc and enkf_archive jobs. Is this intentional to have this change the archive scripts instead of the update scripts? Please confirm, thanks!

@mingshichen-noaa
Copy link
Contributor Author

@KateFriedman-NOAA
This PR (#3291) doesn't include "JGDAS_ENKF_UPDATE" job and associated scripts because previous PR (#3170) CI tests ran into trouble (test status of some tasks failed in Hercules). I am working on a new branch (feature/jgdas_enkf_update_job) in hera environment).

@mingshichen-noaa
Copy link
Contributor Author

mingshichen-noaa commented Feb 3, 2025 •
edited
Loading

@aerorahul, @KateFriedman-NOAA , @WalterKolczynski-NOAA
In order to identify the CI testing failure of the PR (#3291) with ci/cases/pr/ C96C48_hybatmDA.yaml, I tested the develop branch in Hercules and found the testing develop resulted in the same failure (refer to the following table):

   CYCLE                    TASK                       JOBID               STATE         EXIT STATUS     TRIES      DURATION

================================================================================================================================
202112201800 gdas_stage_ic 4395752 SUCCEEDED 0 1 46.0
202112201800 gdas_fcst_seg0 4395779 SUCCEEDED 0 1 328.0
202112201800 gdas_atmos_prod_f000 4395796 SUCCEEDED 0 1 80.0
202112201800 gdas_atmos_prod_f001 4395797 SUCCEEDED 0 1 64.0
202112201800 gdas_atmos_prod_f002 4395798 SUCCEEDED 0 1 64.0
202112201800 gdas_atmos_prod_f003 4395799 SUCCEEDED 0 1 67.0
202112201800 gdas_atmos_prod_f004 4395855 SUCCEEDED 0 1 66.0
202112201800 gdas_atmos_prod_f005 4395856 SUCCEEDED 0 1 64.0
202112201800 gdas_atmos_prod_f006 4395857 SUCCEEDED 0 1 61.0
202112201800 gdas_atmos_prod_f007 4395858 SUCCEEDED 0 1 63.0
202112201800 gdas_atmos_prod_f008 4395859 SUCCEEDED 0 1 64.0
202112201800 gdas_atmos_prod_f009 4395860 SUCCEEDED 0 1 64.0
202112201800 enkfgdas_echgres 4395861 SUCCEEDED 0 1 35.0
202112201800 enkfgdas_fcst_mem001 4395780 SUCCEEDED 0 1 260.0
202112201800 enkfgdas_fcst_mem002 4395781 SUCCEEDED 0 1 257.0
202112201800 enkfgdas_epos000 4395862 DEAD 1 2 15.0
202112201800 enkfgdas_epos001 4395863 DEAD 1 2 15.0
202112201800 enkfgdas_epos002 4395864 DEAD 1 2 13.0
202112201800 enkfgdas_epos003 4395865 DEAD 1 2 15.0
202112201800 enkfgdas_epos004 4395866 DEAD 1 2 14.0
202112201800 enkfgdas_epos005 4395867 DEAD 1 2 14.0
202112201800 enkfgdas_epos006 4395868 DEAD 1 2 14.0
202112201800 enkfgdas_stage_ic 4395753 SUCCEEDED 0 1 44.0

testing results are located at:

  1. develop case:
    /work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC_DEV/EXPDIR/C96C48_hybatmDA
  2. feature/jgdas_enkf_sfc_job case:
    /work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC/EXPDIR/C96C48_hybatmDA

Please give me suggestions. Thanks.

WalterKolczynski-NOAA
WalterKolczynski-NOAA requested changes Feb 3, 2025
View reviewed changes
scripts/exgdas_enkf_sfc.sh Outdated
Comment on lines 168 to 169
${NCP} "${COMOUT_ATMOS_ANALYSIS_MEM}/${APREFIX_ENS}sfci00${FHR}.nc" \
"${DATA}/lnd_incr.${cmem}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an input to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WalterKolczynski-NOAA
in view of comment on line 176, this implies an output on lines 189-190.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't follow 😕.

This is copying from COM (to DATA), so it is an input. If it copies to COM (from DATA), it is an output.

scripts/exgdas_enkf_sfc.sh Outdated
Comment on lines 198 to 199
${NCP} "${COMOUT_ATMOS_ANALYSIS_MEM}/${APREFIX_ENS}sfci00${FHR}.nc" \
"${DATA}/lnd_incr.${cmem}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also looks like an input.

Copy link
Contributor Author

@mingshichen-noaa mingshichen-noaa Feb 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in view of comment on line 176, this implies an output on lines 189-190.

jobs/JGDAS_ENKF_SFC Outdated
COM_ATMOS_ANALYSIS_DET:COM_ATMOS_ANALYSIS_TMPL
RUN=${RUN/enkf} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \
COMIN_OBS:COM_OBS_TMPL \
COMOUT_ATMOS_ANALYSIS_DET:COM_ATMOS_ANALYSIS_TMPL
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Less obvious, but I think this is an input too.

Copy link
Contributor Author

@mingshichen-noaa mingshichen-noaa Feb 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WalterKolczynski-NOAA
I have tested two cases with C96C48_hybatmDA.yaml

  1. COMIN_ATMOS_ANALYSIS_DET:COM_ATMOS_ANALYSIS_TMPL on line 37
  2. COMOUT_ATMOS_ANALYSIS_DET:COM_ATMOS_ANALYSIS_TMPL on line 37
    Testing results showed successful state for all tasks in Hercules. Hence I agree with your suggestion--an input.

@WalterKolczynski-NOAA
Copy link
Contributor

testing results are located at:

  1. develop case:
    /work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC_DEV/EXPDIR/C96C48_hybatmDA
  2. feature/jgdas_enkf_sfc_job case:
    /work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC/EXPDIR/C96C48_hybatmDA

Both of these are using the same HOMEgfs, how is one develop and one your branch? Looks like they both used your branch.

@mingshichen-noaa
Copy link
Contributor Author

@WalterKolczynski-NOAA

  1. develop case:
    At the same HOMEgfs, I used "git switch develop" and did "cd sorc, ./build_all.sh, ./link_workflow.sh, cd ../sorc, and source workflow/gw_setup.sh ", finally did
    pslot=C96C48_hybatmDA HPC_ACCOUNT=fv3-cpu RUNTESTS=/work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC_DEV ICSDIR_ROOT=/work/noaa/global/glopara/data/ICSDIR ./workflow/create_experiment.py --yaml ci/cases/pr/C96C48_hybatmDA.yaml
    Testing results are located at
    /work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC_DEV/EXPDIR/C96C48_hybatmDA

  2. feature/jgdas_enkf_sfc_job case:
    At the same HOMEgfs, I used "git switch feature/jgdas_enkf_sfc_job" and did "cd sorc, ./build_all.sh, ./link_workflow.sh, cd ../sorc, and source workflow/gw_setup.sh ", finally did
    pslot=C96C48_hybatmDA HPC_ACCOUNT=fv3-cpu RUNTESTS=/work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC ICSDIR_ROOT=/work/noaa/global/glopara/data/ICSDIR ./workflow/create_experiment.py --yaml ci/cases/pr/C96C48_hybatmDA.yaml
    Testing results are located at
    /work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SFC/EXPDIR/C96C48_hybatmDA

In above two cases, testing results show dead status for tasks (enkfgdas_epos000 ~ enkfgdas_epos006).

@mingshichen-noaa
Copy link
Contributor Author

@WalterKolczynski-NOAA
I agree with your suggestion and have changed COMOUT_ATMOS_ANALYSIS_MEM into COMIN_ATMOS_ANALYSIS_MEM in scripts/exgdas_enkf_sfc.sh.

@WalterKolczynski-NOAA
Copy link
Contributor

@mingshichen-noaa develop has changed some of these, creating conflicts. Please resolve the conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA requested changes

@aerorahul aerorahul Awaiting requested review from aerorahul

@KateFriedman-NOAA KateFriedman-NOAA Awaiting requested review from KateFriedman-NOAA

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mingshichen-noaa @KateFriedman-NOAA @WalterKolczynski-NOAA