Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ufs-case-studies WE2E tests fail on Derecho in get_extrn_ics/lbcs #1144

Open
MichaelLueken opened this issue Oct 28, 2024 · 0 comments
Open
Labels
bug Something isn't working

Comments

@MichaelLueken
Copy link
Collaborator

Expected behavior

The ufs-case-studies WE2E tests should properly run on Derecho and not fail in the get_extrn_ics/lbcs tasks.

Current behavior

The ufs-case-studies WE2E tests are currently failing intermittently on Derecho in the get_extrn_ics/lbcs tasks due to being unable to find necessary files in S3 bucket.

Machines affected

The ufs-case-studies WE2E tests are only failing on Derecho.

Steps To Reproduce

  1. Clone the SRW App on Derecho
  2. Build the App
  3. Run the ufs-case-studies WE2E tests (./run_WE2E_tests.py -t ufs_case_studies -m derecho -a NRAL0032)
  4. See intermittent failures in the get_extrn_ics/lbcs tasks
@MichaelLueken MichaelLueken added the bug Something isn't working label Oct 28, 2024
MichaelLueken added a commit that referenced this issue Nov 1, 2024
…n-env spack-stack environment (#1136)

* Update ufs-weather-model hash to 38a29a6 (September 19)
* Update UPP hash to 81b38a8 (August 13)
* All Tier-1 modulefiles/build_*.lua files have been updated to use the upp-addon-env spack-stack environment
* srw_common.lua was updated to use g2/3.5.1 and g2tmpl/1.13.0 (these are required for UPP)
* .cicd/JENKINSFILE was updated to replace cheyenne entries with derecho.
* The doc/tables/Tests.csv table had nco-mode WE2E tests removed
* The doc/UsersGuide/CustomizingTheWorkflow/ConfigWorkflow.rst documentation was updated to updated ush/config_defaults.yaml file.
* The .github/CODEOWNERS file was updated to add Bruce Kropp to the list of reviewers
* The exregional_plot_allvars.py and exregional_plot_allvars_diff.py scripts were updated to address changes made to the postxconfig-NT-fv3lam.txt file.
* Updated ush/config_defaults.yaml to update PE_MEMBER01 calculation and documentation for OMP_NUM_THREADS_RUN_FCST to allow for the run_fcst task to properly run on Tier-1 platforms after updates to allow threading to function properly.
* The ush/machine/*.yaml files were updated to allow for the run_fcst task to properly run on Tier-1 platforms after updates to allow threading to function properly.
* There are not enough resources on Jet to run the high resolution WE2E tests (136 (ReqNodeNotAvail)). Commented out the tests in the comprehensive.jet test suite and removed one test from the coverage.jet test suite.
* The ufs-case-studies WE2E tests are currently failing on Derecho. The failure is due to the file not being available. This is an issue because the file in question is named correctly and is available, but the tests fail in the get_extrn_ics/lbs tasks stating that the files aren't present. Commented out these tests in comprehensive.derecho and moved WE2E tests to remove from coverage.derecho. Issue ufs-case-studies WE2E tests fail on Derecho in get_extrn_ics/lbcs #1144 was opened to track this issue on Derecho.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant