Merge release/1.4.1 into develop #671

climbfuji · 2023-07-14T13:13:11Z

Summary

See title. This is pretty straightforward. I made the following change to the documentation after pulling in the release branch: The release branch had "spack-stack-1.4.1 not supported on this platform, please use 1.4.0" for a few selected machines. I replaced this with the original spack-stack-1.4.0 documentation and added a note on top that spack-stack-1.4.1 is not available on all platforms and that for those the documentation lists 1.4.0 further down.

Note. This should be merged as a regular merge, not as a squashed merge.

Testing

Both develop as of today and the release branch have undergone automatic and manual testing. For this PR, CI is sufficient.

Applications affected

This release enables the transition of the UFS to spack-stack, JEDI has only two minor version updates (eckit, bufr) that it will test in preparation for spack-stack-1.5.0.

Systems affected

See documentation update (but older 1.4.0 installations are still available).

Dependencies

waiting on Merge release/1.4.1 into jcsda_emc_spack_stack spack#296

Issue(s) addressed

Closes #637

Checklist

This PR addresses one issue/problem/enhancement, or has a very good reason for not doing so.
These changes have been tested on the affected systems and applications.
All dependency PRs/issues have been resolved and this PR can be merged.

…into HEAD

Update FMS & SCOTCH defaults

* Add libfabric to modules list in acorn/compilers.yaml * Update acorn/packages.yaml (incl. adding ext. flex) * update bufr default -> 12.0.0

JCSDA#669) * Update Hercules site config * Update mirrors for Hercules and Orion * Update configs/sites/gaea-c5/packages.yaml * Replace spack submodule release branch with tag * Update of doc/source/MaintainersSection.rst and doc/source/PreConfiguredSites.rst for spack-stack-1.4.1 * [skip ci] Doc updates for pre-configured sites Acorn and Parallel Works

…into develop

…d testing

climbfuji · 2023-07-14T13:21:25Z

.gitmodules

+  ##branch = develop
+  #url = https://github.com/jcsda/spack
+  #branch = jcsda_emc_spack_stack
+  url = https://github.com/climbfuji/spack


This change is temporary and will be reverted once JCSDA/spack#296 is merged.

configs/common/packages.yaml

climbfuji · 2023-07-14T13:23:28Z

configs/sites/hercules/packages.yaml

@@ -144,7 +144,7 @@ packages:
  qt:
    externals:
    - spec: [email protected]
-      prefix: /apps/spack-managed/gcc-11.3.1/qt-5.15.8-d47tsna6f5dylcpblkfgw4gpn2cucihd
+      prefix: /apps/spack-managed/gcc-12.2.0/qt-5.15.8-gayzhaahclvlybiiykwj2cym4vo33w6x


Note to self, this needs to be reverted. But it has no significance, since qt isn't used in spack-stack directly.

srherbener · 2023-07-14T14:36:34Z

configs/common/packages.yaml

@@ -20,7 +20,7 @@
      version: [1.78.0]
      variants: ~atomic +chrono +date_time +exception +filesystem ~graph ~iostreams ~locale ~log ~math ~mpi ~numpy +pic +program_options +python ~random +regex +serialization ~signals +system +test +thread +timer ~wave cxxstd=14 visibility=hidden
    bufr:
-      version: [11.7.1]
+      version: [12.0.0]


I think I'm good with making [email protected] the default version, but I want to double check. JEDI users should be using spack-stack-1.4.0 which has [email protected] installed. Because of this JEDI users will be insulated from this change so it's okay to make [email protected] the default version on the develop branch.

just a note: [email protected] is producing large differences in GSI regression tests (as reported by @DavidHuber-NOAA: NOAA-EMC/GSI#589 (comment)). fine to merge this here, but i expect there might be some issues w/ some applications during transition to this version.

@ulmononian one thing to check would be if the GSI is currently using the [email protected] bufr_8 component (which is using 8-byte integers and 8-byte floats, ie double precision), or the bufr_d component (which has 4-byte integers and 8-byte floats). See the README.md notes at https://github.com/NOAA-EMC/NCEPLIBS-bufr/tree/bufr_v11.7.1 for details.

The only component available in [email protected] is bufr_4 which is 4-byte integers and 4-byte floats (ie single precision) so the GSI bufr related code could be making a switch from double to single precision.

Let me note that if it turns out that we need both bufr@12 and bufr@11 in spack-stack-1.5.0, we can install both and configure the different environments (gsi-env etc.) such that they load the correct one.

@srherbener Indeed, the GSI uses the bufr_d library, so that could explain the differences. Thanks!

@srherbener thanks for providing that insight, steve. @DavidHuber-NOAA do you expect the GSI team to accept the differences w/ bufr_4 / bufr@12 or should we plan to provide bufr@11 for gsi purposes for the foreseeable future?

@ulmononian I don't expect that they will accept it or at least that they will want to run some more testing before making the switch. Also, I don't see any hints on their GitHub page that they desire to upgrade at this time. But I have asked the question -- and will ask again on a different channel -- and will update when I hear back.

climbfuji · 2023-07-14T18:24:53Z

I'll go ahead and merge this. @DavidHuber-NOAA Please keep us posted if GSI needs bufr@11 (in addition to the existing bufr@12) in the next spack-stack-1.5.0 release (planned for early September).

AlexanderRichert-NOAA and others added 18 commits June 20, 2023 09:31

set build_type=Release for fms

b517acf

update scotch variants in configs/common/packages.yaml

ef0f3d7

update .gitmodules

53de65a

submodule pointer

f10f1e3

submodule pointer

c3a4140

update gitmodules

35298b1

submodule pointer

1e1c2a4

remove tab

2181c7e

fix gitmodules file

45e96ce

Update eckit to 1.23.1 (JCSDA#635)

928ee77

Merge branch 'release/1.4.1' of https://github.com/jcsda/spack-stack …

248628a

…into HEAD

Merge pull request JCSDA#632 from AlexanderRichert-NOAA/ufs_jun2023

3322236

Update FMS & SCOTCH defaults

Update Acorn config and include [email protected] for 1.4.1 (JCSDA#636)

17e75be

* Add libfabric to modules list in acorn/compilers.yaml * Update acorn/packages.yaml (incl. adding ext. flex) * update bufr default -> 12.0.0

update container README (JCSDA#639)

6ffd4e7

update submodule pointer for scotch mpi patch (JCSDA#660)

8475534

Merge branch 'release/1.4.1' of https://github.com/jcsda/spack-stack …

7e50533

…into develop

Update .gitmodules and submodule pointer for spack for code review an…

c429794

…d testing

This was referenced Jul 14, 2023

Merge release/1.4.1 into jcsda_emc_spack_stack JCSDA/spack#296

Merged

Roll out spack-stack-1.4.1 bug fix release #637

Closed

climbfuji added the INFRA JEDI Infrastructure label Jul 14, 2023

climbfuji self-assigned this Jul 14, 2023

climbfuji commented Jul 14, 2023

View reviewed changes

configs/common/packages.yaml Show resolved Hide resolved

climbfuji commented Jul 14, 2023

View reviewed changes

climbfuji marked this pull request as ready for review July 14, 2023 13:24

climbfuji requested review from srherbener and AlexanderRichert-NOAA and removed request for srherbener and AlexanderRichert-NOAA July 14, 2023 13:24

climbfuji requested review from srherbener, ulmononian, AlexanderRichert-NOAA and natalie-perlin July 14, 2023 13:24

climbfuji mentioned this pull request Jul 14, 2023

New configurations for Derecho #668

Merged

3 tasks

srherbener reviewed Jul 14, 2023

View reviewed changes

srherbener approved these changes Jul 14, 2023

View reviewed changes

ulmononian approved these changes Jul 14, 2023

View reviewed changes

[skip ci] Revert .gitmodules and update submodule pointer for spack

93e2a71

climbfuji merged commit ae6dacc into JCSDA:develop Jul 14, 2023

climbfuji deleted the feature/merge_141_into_jcsda_emc_spack_stack branch July 14, 2023 18:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge release/1.4.1 into develop #671

Merge release/1.4.1 into develop #671

climbfuji commented Jul 14, 2023 •

edited

Loading

climbfuji Jul 14, 2023

climbfuji Jul 14, 2023

srherbener Jul 14, 2023

climbfuji Jul 14, 2023

ulmononian Jul 14, 2023

srherbener Jul 14, 2023

climbfuji Jul 14, 2023

DavidHuber-NOAA Jul 14, 2023

ulmononian Jul 14, 2023

DavidHuber-NOAA Jul 14, 2023

climbfuji commented Jul 14, 2023

Merge release/1.4.1 into develop #671

Merge release/1.4.1 into develop #671

Conversation

climbfuji commented Jul 14, 2023 • edited Loading

Summary

Testing

Applications affected

Systems affected

Dependencies

Issue(s) addressed

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

climbfuji commented Jul 14, 2023

climbfuji commented Jul 14, 2023 •

edited

Loading