Merge branch 'develop' into feature/jedi_ci_container_and_fms_update_…

…20240502
JCSDA · May 9, 2024 · e64a316 · e64a316
2 parents 3ad6d8a + 8c22669
commit e64a316
Show file tree

Hide file tree

Showing 5 changed files with 84 additions and 51 deletions.
diff --git a/configs/sites/noaa-aws/README.md b/configs/sites/noaa-aws/README.md
@@ -1,16 +1,19 @@
-This README provides step by step instructions for installing the basic packages
-(OS packages, external packages) for spack-stack. Following these steps ensures
-that the site configuration files in `configs/sites/noaa-aws` work out of the box.
+# Provisiong ParallelWorks AWS clusters
 
-# Basic system packages (need to be installed each time a cluster is spun up)
+## Steps to perform when provisioning a cluster for the very first time
 
+This section provides step by step instructions for installing the basic packages (OS packages, external packages) for spack-stack. Following these steps ensures that the site configuration files in `configs/sites/noaa-aws` work out of the box.
+
+### Basic system packages
+```
 sudo su
 chmod 777 /contrib
+
 # The following three commands were necessary to fix failures accessing these repos
 yum-config-manager --disable intel-clck-2019-repo
 yum-config-manager --disable intel-hpc-platform
 yum-config-manager --disable intelpython
-#
+
 yum install -y qt5-qtbase-devel
 yum install -y qt5-qtsvg-devel
 yum install -y xorg-x11-xauth
@@ -19,9 +22,9 @@ yum install -y perl-IPC-Cmd
 yum install -y gettext-devel
 yum install -y m4
 exit
-
-# Create a script that can be added to the cluster resource config so that these packages get installed automatically
-
+```
+Create a script that can be added to the cluster resource config so that these packages get installed automatically when provisioning new clusters later:
+```
 mkdir -p /contrib/admin
 cat <<EOF > /contrib/admin/basic_setup.sh
 #!/bin/bash
@@ -37,9 +40,10 @@ yum install -y m4
 EOF
 
 chmod a+x /contrib/admin/basic_setup.sh
+```
 
-# Create a mysql config for local R2D2 use (if applicable)
-
+### Create a mysql config for local R2D2 use (if applicable)
+```
 sudo su
 cat <<EOF > /contrib/admin/my.cnf
 [mysqld]
@@ -54,9 +58,10 @@ pid-file=/mysql_local/run/mariadb.pid
 EOF
 chmod 644 /contrib/admin/my.cnf
 exit
+```
 
-# Build external packages for spack-stack
-
+### Build external packages for spack-stack
+```
 mkdir -p /contrib/spack-stack
 mkdir /contrib/spack-stack/modulefiles
 cd /contrib/spack-stack/
@@ -129,10 +134,16 @@ make install 2>&1 | tee log.install
 cd /contrib/spack-stack/modulefiles
 mkdir ecflow
 # Create the modulefile from the template in doc/modulefile_templates
+```
 
-############## Steps to perform when starting a new cluster ##############
+## Steps to perform when starting a new cluster
 
-source /contrib/admin/basic_setup.sh  # sudo privileges requred to install packages
+This should be done automatically, but doesn't hurt to run again just in case:
+```
+sudo /contrib/admin/basic_setup.sh
+```
+Configure `git` and `aws` command line utilities
+```
 module unuse /opt/cray/craype/default/modulefiles
 module unuse /opt/cray/modulefiles
 module use /contrib/spack-stack/modulefiles
@@ -150,3 +161,4 @@ mkdir ~/.aws
 # Create ~/.aws/config
 # Create ~/.aws/credentials
 chmod 400 ~/.aws/credentials
+```
diff --git a/configs/sites/noaa-azure/README.md b/configs/sites/noaa-azure/README.md
@@ -1,17 +1,19 @@
-This README provides step by step instructions for installing the basic packages
-(OS packages, external packages) for spack-stack. Following these steps ensures
-that the site configuration files in `configs/sites/noaa-azure` work out of the box.
+# Provisiong ParallelWorks Azure clusters
 
-# Basic system packages (need to be installed each time a cluster is spun up)
+## Steps to perform when provisioning a cluster for the very first time
 
+This section provides step by step instructions for installing the basic packages (OS packages, external packages) for spack-stack. Following these steps ensures that the site configuration files in `configs/sites/noaa-aws` work out of the box.
+
+### Basic system packages
+```
 sudo su
 chmod 777 /contrib
+
 # The following three commands were necessary to fix failures accessing these repos
 yum-config-manager --disable intel-clck-2019-repo
 yum-config-manager --disable intel-hpc-platform
 yum-config-manager --disable intelpython
-#
-yum install -y m4
+
 yum install -y qt5-qtbase-devel
 yum install -y qt5-qtsvg-devel
 yum install -y xorg-x11-xauth
@@ -20,10 +22,11 @@ yum install -y perl-IPC-Cmd
 yum install -y gettext-devel
 yum install -y ncurses-devel
 yum install -y ncurses-static
+yum install -y m4
 exit
-
-# Create a script that can be added to the cluster resource config so that these packages get installed automatically
-
+```
+Create a script that can be added to the cluster resource config so that these packages get installed automatically when provisioning new clusters later:
+```
 mkdir -p /contrib/admin
 cat <<EOF > /contrib/admin/basic_setup.sh
 #!/bin/bash
@@ -41,9 +44,10 @@ yum install -y m4
 EOF
 
 chmod a+x /contrib/admin/basic_setup.sh
+```
 
-# Create a mysql config for local R2D2 use (if applicable)
-
+### Create a mysql config for local R2D2 use (if applicable)
+```
 sudo su
 cat <<EOF > /contrib/admin/my.cnf
 [mysqld]
@@ -58,9 +62,10 @@ pid-file=/mysql_local/run/mariadb.pid
 EOF
 chmod 644 /contrib/admin/my.cnf
 exit
+```
 
-# Build external packages for spack-stack
-
+### Build external packages for spack-stack
+```
 mkdir -p /contrib/spack-stack
 mkdir /contrib/spack-stack/modulefiles
 cd /contrib/spack-stack/
@@ -133,18 +138,23 @@ make install 2>&1 | tee log.install
 cd /contrib/spack-stack/modulefiles
 mkdir ecflow
 # Create the modulefile from the template in doc/modulefile_templates
+```
 
-############## Steps to perform when starting a new cluster ##############
+## Steps to perform when starting a new cluster
 
-source /contrib/admin/basic_setup.sh  # sudo privileges requred to install packages
+This should be done automatically, but doesn't hurt to run again just in case:
+```
+sudo /contrib/admin/basic_setup.sh
+```
+Configure `git` and `aws` command line utilities
+```
 module unuse /opt/cray/craype/default/modulefiles
 module unuse /opt/cray/modulefiles
 module use /contrib/spack-stack/modulefiles
 module load cmake/3.27.2
 module load ecflow/5.8.4
 module load mysql/8.0.31
 module load git-lfs/2.4.1
-#
 
 git lfs install
 git config --global credential.helper cache # or store
@@ -155,3 +165,4 @@ mkdir ~/.aws
 # Create ~/.aws/config
 # Create ~/.aws/credentials
 chmod 400 ~/.aws/credentials
+```
diff --git a/configs/sites/noaa-gcloud/README.md b/configs/sites/noaa-gcloud/README.md
@@ -1,16 +1,19 @@
-This README provides step by step instructions for installing the basic packages
-(OS packages, external packages) for spack-stack. Following these steps ensures
-that the site configuration files in `configs/sites/noaa-gcloud` work out of the box.
+# Provisiong ParallelWorks Gcloud clusters
 
-# Basic system packages (need to be installed each time a cluster is spun up)
+## Steps to perform when provisioning a cluster for the very first time
 
+This section provides step by step instructions for installing the basic packages (OS packages, external packages) for spack-stack. Following these steps ensures that the site configuration files in `configs/sites/noaa-aws` work out of the box.
+
+### Basic system packages
+```
 sudo su
 chmod 777 /contrib
+
 # The following three commands were necessary to fix failures accessing these repos
 yum-config-manager --disable intel-clck-2019-repo
 yum-config-manager --disable intel-hpc-platform
 yum-config-manager --disable intelpython
-#
+
 yum install -y qt5-qtbase-devel
 yum install -y qt5-qtsvg-devel
 yum install -y xorg-x11-xauth
@@ -20,9 +23,9 @@ yum install -y gettext-devel
 yum install -y m4
 yum install -y finger
 exit
-
-# Create a script that can be added to the cluster resource config so that these packages get installed automatically
-
+```
+Create a script that can be added to the cluster resource config so that these packages get installed automatically when provisioning new clusters later:
+```
 mkdir -p /contrib/admin
 cat <<EOF > /contrib/admin/basic_setup.sh
 #!/bin/bash
@@ -42,15 +45,10 @@ yum install -y finger
 EOF
 
 chmod a+x /contrib/admin/basic_setup.sh
+```
 
-# Enable R2D2 experiment scrubber in cron (if applicable)
-
-Refer to https://github.com/JCSDA-internal/jedi-tools/tree/develop/crontabs/noaa-gcloud
-
-The scripts are all set up in the /contrib space and should work after a restart of the cluster. However, any updates to R2D2 that require changes to the scrubber scripts need to be made!
-
-# Create a mysql config for local R2D2 use (if applicable)
-
+### Create a mysql config for local R2D2 use (if applicable)
+```
 sudo su
 cat <<EOF > /contrib/admin/my.cnf
 [mysqld]
@@ -65,9 +63,10 @@ pid-file=/mysql_local/run/mariadb.pid
 EOF
 chmod 644 /contrib/admin/my.cnf
 exit
+```
 
-# Build external packages for spack-stack
-
+### Build external packages for spack-stack
+```
 mkdir -p /contrib/spack-stack
 mkdir /contrib/spack-stack/modulefiles
 cd /contrib/spack-stack/
@@ -140,10 +139,16 @@ make install 2>&1 | tee log.install
 cd /contrib/spack-stack/modulefiles
 mkdir ecflow
 # Create the modulefile from the template in doc/modulefile_templates
+```
 
-############## Steps to perform when starting a new cluster ##############
+## Steps to perform when starting a new cluster
 
-source /contrib/admin/basic_setup.sh  # sudo privileges requred to install packages
+This should be done automatically, but doesn't hurt to run again just in case:
+```
+sudo /contrib/admin/basic_setup.sh
+```
+Configure `git` and `aws` command line utilities
+```
 module unuse /opt/cray/craype/default/modulefiles
 module unuse /opt/cray/modulefiles
 module use /contrib/spack-stack/modulefiles
@@ -161,3 +166,4 @@ mkdir ~/.aws
 # Create ~/.aws/config
 # Create ~/.aws/credentials
 chmod 400 ~/.aws/credentials
+```
diff --git a/doc/source/KnownIssues.rst b/doc/source/KnownIssues.rst
@@ -35,6 +35,10 @@ General
 
    This error usually indicates that the wrong module type is used in the ``spack module ... refresh`` command. For example, the system is configured for ``lmod``, but the command used is ``spack module tcl refresh``.
 
+8. Runtime segmentation faults for applications with ``intel@2021.10.0``: ``Relink `/opt/intel/oneapi/compiler/2024.0/lib/libirc.so' with `/lib/x86_64-linux-gnu/libc.so.6' for IFUNC symbol `memmove' - Segmentation fault (core dumped)``.
+
+   This problem is caused by a bad library in the Intel oneAPI installation. The solution is to fix the library using patchelf, which requires write access to the oneAPI installation: First, verify that ``ldd /opt/intel/oneapi/compiler/2024.0/lib/libirc.so`` says it is statically linked (it isn't). Then, run: ``patchelf --add-needed libc.so.6 /opt/intel/oneapi/compiler/2024.0/lib/libirc.so`` and your application should run rightaway (no need to recompile).
+
 ==============================
 MSU Hercules
 ==============================

diff --git a/spack b/spack