linaro-forge 23.1.2 (#71)

* update linaro-forge to 23.1.2
* deploy it on eiger and santis
* update docs to reflect new uenv features (uenv image pull, etc)

README.md

@@ -1 +1,5 @@
-# Spack stacks
+# Alps Uenv Recipes
+
+This repository manages the recipes for uenv on CSCS' Alps clusters, and the definition of the CI/CD pipeline that deploys them.
+
+See the [documenation](https://eth-cscs.github.io/alps-uenv/) for an overview of the uenv, and a packaging guide.

config.yaml

@@ -71,14 +71,15 @@ uenvs:
         eiger: [zen2]
       develop: False
   linaro-forge:
-    "23.0.3":
+    "23.1.2":
       recipes:
-        zen2: "23.0.3"
-        zen3: "23.0.3"
-        a100: "23.0.3"
+        zen2: "23.1.2"
+        zen3: "23.1.2"
+        a100: "23.1.2"
+        gh200: "23.1.2"
       deploy:
         eiger: [zen2]
-        clariden: [a100]
+        santis: [gh200]
       develop: False
       mount: "/user-tools"
   mch:

docs/uenv-linaro-forge.md

@@ -1,124 +1,213 @@
 # Linaro Forge (DDT) debugger
 
-- https://www.linaroforge.com/downloadForge
-
-Linaro Forge (formerly known as DDT) allows source-level debugging of Fortran,
+[Linaro Forge](https://www.linaroforge.com/downloadForge) (formerly known as DDT) allows source-level debugging of Fortran,
 C, C++ and Python codes. It can be used for debugging serial, multi-threaded
 (OpenMP), multi-process (MPI) and accelerated (Cuda, OpenACC) programs running
 on research and production systems, including CSCS Alps system. It can be
 executed either as a graphical user interface or from the command-line.
 
-## Using the debugger
+## Usage notes
+
+The uenv is named `linaro-forge`, and the available versions on a cluster can be determined using the `uenv image find` command, for example:
+```
+> uenv image find linaro-forge
+uenv/version:tag                        uarch date       id               size
+linaro-forge/23.1.2:latest              gh200 2024-04-10 ea67dbb33801c7c3 342MB
+```
+
+
+The linaro tools are configured to be mounted in the `/user-tools` path so that they can be used alongside application and development uenv mounted at `user-environment`.
+
+=== "sidecar"
+
+    When using alongside another uenv, start a uenv session with both uenv with `linaro-forge` after the main uenv, to mount the images at the respective `/user-environment` and `/user-tools` locations:
+
+    ```bash
+    uenv start prgenv-gnu/24.2:v3 linaro-forge/32.1.2
+
+    # test that everything has been mounted correctly
+    # (will give warnings if there are problems)
+    uenv status
+
+    uenv view prgenv-gnu:default
+    source /user-tools/acvitave
+
+    # check that ddt is in the path
+    ddt --version
+    ```
+
+    The `/user-tools/activate` script will make the forge executables available in your environment, and **must be run after** any other uenv view command.
+
+=== "standalone"
+
+    When using the uenv with no other environment mounted, you will need to explicitly set the `/user-tools` mount point:
+
+    ```bash
+    uenv start linaro-forge/32.1.2:/user-tools
+
+    source /user-tools/acvitave
+
+    # check that ddt is in the path
+    ddt --version
+    ```
+
+    The `/user-tools/activate` script will make the forge executables available in your environment.
+
+## Getting Started
 
 In order to debug your code on Alps, you need to:
 
-- install the Forge/DDT client on your laptop,
-- setup the user environment on Alps,
-- build an executable with debug flags on Alps,
+1. pull the linaro-forge uenv on the target Alps vCluster
+- install the Forge/DDT client on your laptop
+- build an executable with debug flags
 - launch a job with the debugger on Alps.
 
+### Pull the Linaro Forge uenv on the Alps cluster
+
+The first step is to download the latest version of linaro-forge that is available on the cluster.
+First, SSH into the target system, then use the `uenv image find` command to list the available versions on the system:
+
+```
+> uenv image find linaro-forge
+uenv/version:tag                        uarch date       id               size
+linaro-forge/23.1.2:latest              gh200 2024-04-10 ea67dbb33801c7c3 342MB
+```
+
+In this example, there is a single version available. Next we pull the image so that it is available locally.
+```
+> uenv image pull linaro-forge/23.1.2:latest
+```
+
+It will take a few seconds to download the image. Once complete, check that it was downloaded using the `uenv image ls` command:
+
+```
+> uenv image ls linaro-forge
+uenv/version:tag                        uarch date       id               size
+linaro-forge/23.1.2:latest              gh200 2024-04-05 ea67dbb33801c7c3 342MB
+```
 
 ### Install the client on your laptop
 
-We recommend to download and install the [desktop client](https://www.linaroforge.com/downloadForge) on your local workstation/laptop. It will connect with the debug jobs running on Alps, offering a better user experience compared to opening ddt with X11 forwarding. The client can be downloaded for a selection of operating systems.
+We recommend installing the [desktop client](https://www.linaroforge.com/downloadForge) on your local workstation/laptop.
+It can be configured to connect with the debug jobs running on Alps, offering a better user experience compared running remotely with X11 forwarding.
+The client can be downloaded for a selection of operating systems, via the link above.
 
-Once installed, the client needs to be configured to connect to your preferred vcluster. For this, launch the client:
+Once installed, the client needs to be configured to connect to the vCluster on which you are working.
+First, start the client on your laptop.
 
-- mac: open /Applications/Linaro\ Forge\ Client\ 23.0.1.app/
-- linux: $HOME/linaro/forge/23.0.1/bin/ddt
+=== "Linux"
 
-and setup the connection:
+    The path will change if you have installed a different version, or if it has been installed in a non-standard installation location.
 
-```
-- open the 'Remote Launch' menu and click on 'configure' then 'Add' and set the fields, for example:
-    - Connection Name: alps
+    ```bash
+    $HOME/linaro/forge/23.0.1/bin/ddt
+    ```
 
-    - Host Name: [email protected] [email protected]
-    # Note that the clariden vlcuster name can be replaced with another vcluster name
+=== "MacOS"
 
-    - Remote install dir: uenv run IMG -- DDTDIR
-      # here we tell the client to use the ddt installed in the uenv image
-```
+    The path will change if you have installed a different version, or if it has been installed in a non-standard installation location.
 
-where you can replace `IMG` and `DDTDIR` with for example:
+    ```bash
+    open /Applications/Linaro\ Forge\ Client\ 23.0.1.app/
+    ```
 
-- `IMG`: full path to the uenv file and mount point, for example:
-  - _/scratch/e1000/your-cscs-username-here/linaro-forge-23.0.3.squashfs:/user-tools_
-- `DDTDIR`: full path to the tool, for example:
-  - _/user-tools/linux-sles15-zen2/gcc-11.3.0/linaro-forge-23.0.3-3z4k6ijkcxcgqymv6mapv6xaela7m2q5/_
+Next, configure a connection to the target system.
+Open the *Remote Launch* menu and click on *configure* then *Add*. Examples of the settings are below.
 
-and
+=== "Eiger"
 
-```
-    - Remote Script:
+    | Field       | Value                                   |
+    | ----------- | --------------------------------------- |
+    | Connection  | `eiger`                                |
+    | Host Name   | `[email protected] [email protected]`  |
+    | Remote Installation Directory | `uenv run linaro-forge/23.1.2:/user-tools -- /user-tools/env/forge/` |
+    | Private Key | `$HOME/.ssh/cscs-key` |
 
-    - Private Key: _path-to-your-home_/.ssh/cscs-key
+=== "Santis"
 
-    - Proxy through login node: yes (check the box)
-```
+    | Field       | Value                                   |
+    | ----------- | --------------------------------------- |
+    | Connection  | `santis`                                |
+    | Host Name   | `[email protected] [email protected]`  |
+    | Remote Installation Directory | `uenv run linaro-forge/23.1.2:/user-tools -- /user-tools/env/forge/` |
+    | Private Key | `$HOME/.ssh/cscs-key` |
 
-Click `Test Remote Launch`. If the client can connect, you are ready to debug:
-click on `ok` and `close` (to save the configuration). You can now connect by going to `Remote Launch` and choose the `Alps` entry. If the client fails to connect, look at the message, check your ssh configuration and make sure you can ssh without the client.
 
-### Setup the environment
+Some notes on the examples above:
 
-`linaro-forge-23.0.3.squashfs` provides the latest version of Linaro Forge (23.0.3).
+* SSH Forwarding via `ela.scscs.ch` is used to access the cluster.
+* the replace the username `bsmith` with your CSCS user name that you would normally use to open an SSH connection to CSCS.
+* the Remote Installation Path is a little bit more complicated than
+* the private keys should be the ones generated for CSCS MFA, and this field does not need to be set if you have added the key to your SSH agent.
 
-- On Alps:
-```bash
-uenv start ./linaro-forge-23.0.3.squashfs
-uenv modules use
-module load linaro-forge
-ddt --version
-# Version: 23.0.3
-```
+Once configured, test and save the configuration:
+
+1. check whether the concfiguration is correct, click `Test Remote Launch`.
+2. Click on `ok` and `close` to save the configuration.
+3. You can now connect by going to `Remote Launch` and choose the `Alps` entry. If the client fails to connect, look at the message, check your ssh configuration and make sure you can ssh without the client.
+
+### Setup the environment
 
 ### Build with debug flags
 
 Once the uenv is loaded and activated, the program to debug must be compiled with the `-g` (for cpu) and `-G` (for gpu) debugging flags. For example, let's build a cuda code with  a user environment:
 
-- on Alps:
 ```bash
-uenv start store.squashfs
-uenv modules use
-module load gcc cray-mpich cuda
-git clone -b ddt https://github.com/jgphpc/octree-miniapp \
-octree-miniapp.git
+uenv start prgenv-gnu:24.2:v2
+uenv view default
+
+# download the source code
+git clone https://github.com/sekelle/octree-miniapp.git
+cd o
+
+
+# build the application
 make -C octree-miniapp.git/
 ```
 
 ### Launch the code with the debugger
 
-Given the unusual way of loading the uenv, the DDT client must be launched in `Manual Launch` mode (assuming that it is connected to Alps via `Remote Launch`):
+To use the DDT client with uenv, it must be launched in `Manual Launch` mode (assuming that it is connected to Alps via `Remote Launch`):
 
-- on the client:
-```
-- open the 'Manual Launch' menu and
-- set the fields, for example:
-    - Number of processes: 12
-    - CUDA: yes (check the box for gpu exeutables)
-```
-Listen and wait 
+??? note
 
-You can then launch ddt with the srun command (or a Slurm jobscript):
+    the steps below do not manually launch - instead they directly launch using `ddt --connect srun ...` on the target cluster.
 
-- on Alps:
-```bash
-unset CUDA_VISIBLE_DEVICES
-srun --uenv=$UENV_SQFS,TOOL_SQFS \
--l -N3 -n12 -t10 -pnvgpu \
-./octree-miniapp.git/cuda_visible_devices.sh \
-$DDT_CLIENT
-./octree-miniapp.git/neighbor_search.exe 120000
-```
+=== "on laptop"
+
+    Start DDT, and connect to the target cluster using the drop down menu for Remote Launch.
+
+    Then wait for the job to start (see the "on Alps" tab).
+
+=== "on Alps"
+
+    log into the system and launch with the srun command:
+
+    ```bash
+    # start a session with both the PE used to build your application
+    # and the linaro-forge uenv mounted
+    uenv start prgenv-gnu/24.2 linaro-forge/23.1.2
+    ddt --connect srun -n2 -N2 ./a.out
+    ```
+
+Notes on using specific systems:
+
+=== "santis"
+
+    !!! warning
+
+        Because Santis is not connected to the internet, some environment variables need to be set so that it can connect to the license server.
 
-where for example:
+        ```bash
+        export https_proxy=proxy.cscs.ch:8080
+        export http_proxy=proxy.cscs.ch:8080
+        export no_proxy=".local, .cscs.ch, localhost, 148.187.0.0/16, 10.0.0.0/8, 172.16.0.0/12"
+        ```
 
-- UENV_SQFS=$PWD/store.squashfs:/user-environment
-- TOOL_SQFS=$PWD/linaro-forge-23.0.3.squashfs:/user-tools
-- DDT_CLIENT=/user-tools/linux-sles15-zen2/gcc-11.3.0/linaro-forge-23.0.3-3z4k6ijkcxcgqymv6mapv6xaela7m2q5/bin/ddt-client
+        ???- note "default value of `http_proxy`"
 
+            By default the `https_proxy` and `http_proxy` variables are set to `http://proxy.cscs.ch:8080`, as the transport is required for some other services to work. You will have to set them for a debugging session.
 
 This screenshot shows a debugging session on 12 gpus:
 
-![DDT](https://raw.githubusercontent.com/jgphpc/octree-miniapp/ddt/img/ddt.png)
+![DDT](https://raw.githubusercontent.com/jgphpc/octree-miniapp/ddt/img/ddt.png)

mkdocs.yml

@@ -5,6 +5,7 @@ nav:
     - 'gromacs': uenv-gromacs.md
     - 'quantum-espresso': uenv-qe.md
     - 'cp2k': uenv-cp2k.md
+    - 'linaro-forge': uenv-linaro-forge.md
 theme:
   name: material
   features:
@@ -21,6 +22,8 @@ markdown_extensions:
       anchor_linenums: true
       line_spans: __span
       pygments_lang_class: true
+  - pymdownx.tabbed:
+      alternate_style: true
   - tables
   - toc:
       permalink: true

recipes/linaro-forge/23.0.3/environments.yaml

@@ -6,4 +6,4 @@ linaro:
         spec: gcc
   unify: true
   specs:
-  - linaro-forge +accept-eula
+  - linaro-forge@23.0.3 +accept-eula

recipes/linaro-forge/23.0.3/repo

@@ -0,0 +1 @@
+../repo

recipes/linaro-forge/23.1.2/compilers.yaml

@@ -0,0 +1,5 @@
+bootstrap:
+  spec: gcc@12
+gcc:
+  specs:
+  - gcc@12

recipes/linaro-forge/23.1.2/config.yaml

@@ -0,0 +1,6 @@
+name: linaro
+store: /user-tools
+spack:
+    repo: https://github.com/spack/spack.git
+    commit: releases/v0.21
+modules: True

recipes/linaro-forge/23.1.2/environments.yaml

@@ -0,0 +1,10 @@
+linaro:
+  compiler:
+      - toolchain: gcc
+        spec: gcc
+  unify: true
+  specs:
+  - [email protected] +accept-eula
+  views:
+    forge:
+      link: roots

recipes/linaro-forge/23.1.2/modules.yaml

@@ -0,0 +1,23 @@
+modules:
+  # Paths to check when creating modules for all module sets
+  prefix_inspections:
+    bin:
+      - PATH
+    lib:
+      - LD_LIBRARY_PATH
+    lib64:
+      - LD_LIBRARY_PATH
+
+  default:
+    arch_folder: false
+    # Where to install modules
+    roots:
+      tcl: /user-environment/modules
+    tcl:
+      all:
+        autoload: none
+      hash_length: 0
+      exclude_implicits: true
+      exclude: ['%[email protected]', 'gcc %[email protected]']
+      projections:
+        all: '{name}/{version}'

recipes/linaro-forge/23.1.2/post-install

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+#forge_bin=$(spack -C {{ env.config }} location -i linaro-forge)/bin
+#ln -s $forge_bin {{ env.mount }}/bin
+forge_bin={{ env.mount }}/env/forge/bin
+echo "export PATH=$forge_bin:$PATH" >> {{ env.mount }}/activate
+echo "export UENV_DDT_VERSION=23.1.2" >> {{ env.mount }}/activate