linaro-forge (#53)

config.yaml

@@ -64,6 +64,17 @@ uenvs:
       deploy:
         eiger: [zen2]
       develop: False
+  linaro-forge:
+    "23.0.3":
+      recipes:
+        zen2: "23.0.3"
+        zen3: "23.0.3"
+        a100: "23.0.3"
+      deploy:
+        eiger: [zen2]
+        clariden: [a100]
+      develop: False
+      mount: "/user-tools"
   mch:
     # v5 was deprecated from the build config due to an out of date
     # version of spack (v0.19)

docs/uenv-linaro-forge.md

@@ -0,0 +1,124 @@
+# Linaro Forge (DDT) debugger
+
+- https://www.linaroforge.com/downloadForge
+
+Linaro Forge (formerly known as DDT) allows source-level debugging of Fortran,
+C, C++ and Python codes. It can be used for debugging serial, multi-threaded
+(OpenMP), multi-process (MPI) and accelerated (Cuda, OpenACC) programs running
+on research and production systems, including CSCS Alps system. It can be
+executed either as a graphical user interface or from the command-line.
+
+## Using the debugger
+
+In order to debug your code on Alps, you need to:
+
+- install the Forge/DDT client on your laptop,
+- setup the user environment on Alps,
+- build an executable with debug flags on Alps,
+- launch a job with the debugger on Alps.
+
+
+### Install the client on your laptop
+
+We recommend to download and install the [desktop client](https://www.linaroforge.com/downloadForge) on your local workstation/laptop. It will connect with the debug jobs running on Alps, offering a better user experience compared to opening ddt with X11 forwarding. The client can be downloaded for a selection of operating systems.
+
+Once installed, the client needs to be configured to connect to your preferred vcluster. For this, launch the client:
+
+- mac: open /Applications/Linaro\ Forge\ Client\ 23.0.1.app/
+- linux: $HOME/linaro/forge/23.0.1/bin/ddt
+
+and setup the connection:
+
+```
+- open the 'Remote Launch' menu and click on 'configure' then 'Add' and set the fields, for example:
+    - Connection Name: alps
+
+    - Host Name: [email protected] [email protected]
+    # Note that the clariden vlcuster name can be replaced with another vcluster name
+
+    - Remote install dir: uenv run IMG -- DDTDIR
+      # here we tell the client to use the ddt installed in the uenv image
+```
+
+where you can replace `IMG` and `DDTDIR` with for example:
+
+- `IMG`: full path to the uenv file and mount point, for example:
+  - _/scratch/e1000/your-cscs-username-here/linaro-forge-23.0.3.squashfs:/user-tools_
+- `DDTDIR`: full path to the tool, for example:
+  - _/user-tools/linux-sles15-zen2/gcc-11.3.0/linaro-forge-23.0.3-3z4k6ijkcxcgqymv6mapv6xaela7m2q5/_
+
+and
+
+```
+    - Remote Script:
+
+    - Private Key: _path-to-your-home_/.ssh/cscs-key
+
+    - Proxy through login node: yes (check the box)
+```
+
+Click `Test Remote Launch`. If the client can connect, you are ready to debug:
+click on `ok` and `close` (to save the configuration). You can now connect by going to `Remote Launch` and choose the `Alps` entry. If the client fails to connect, look at the message, check your ssh configuration and make sure you can ssh without the client.
+
+### Setup the environment
+
+`linaro-forge-23.0.3.squashfs` provides the latest version of Linaro Forge (23.0.3).
+
+- On Alps:
+```bash
+uenv start ./linaro-forge-23.0.3.squashfs
+uenv modules use
+module load linaro-forge
+ddt --version
+# Version: 23.0.3
+```
+
+### Build with debug flags
+
+Once the uenv is loaded and activated, the program to debug must be compiled with the `-g` (for cpu) and `-G` (for gpu) debugging flags. For example, let's build a cuda code with  a user environment:
+
+- on Alps:
+```bash
+uenv start store.squashfs
+uenv modules use
+module load gcc cray-mpich cuda
+git clone -b ddt https://github.com/jgphpc/octree-miniapp \
+octree-miniapp.git
+make -C octree-miniapp.git/
+```
+
+### Launch the code with the debugger
+
+Given the unusual way of loading the uenv, the DDT client must be launched in `Manual Launch` mode (assuming that it is connected to Alps via `Remote Launch`):
+
+- on the client:
+```
+- open the 'Manual Launch' menu and
+- set the fields, for example:
+    - Number of processes: 12
+    - CUDA: yes (check the box for gpu exeutables)
+```
+Listen and wait 
+
+You can then launch ddt with the srun command (or a Slurm jobscript):
+
+- on Alps:
+```bash
+unset CUDA_VISIBLE_DEVICES
+srun --uenv=$UENV_SQFS,TOOL_SQFS \
+-l -N3 -n12 -t10 -pnvgpu \
+./octree-miniapp.git/cuda_visible_devices.sh \
+$DDT_CLIENT
+./octree-miniapp.git/neighbor_search.exe 120000
+```
+
+where for example:
+
+- UENV_SQFS=$PWD/store.squashfs:/user-environment
+- TOOL_SQFS=$PWD/linaro-forge-23.0.3.squashfs:/user-tools
+- DDT_CLIENT=/user-tools/linux-sles15-zen2/gcc-11.3.0/linaro-forge-23.0.3-3z4k6ijkcxcgqymv6mapv6xaela7m2q5/bin/ddt-client
+
+
+This screenshot shows a debugging session on 12 gpus:
+
+![DDT](https://raw.githubusercontent.com/jgphpc/octree-miniapp/ddt/img/ddt.png)

recipes/linaro-forge/23.0.3/compilers.yaml

@@ -0,0 +1,5 @@
+bootstrap:
+  spec: gcc@12
+gcc:
+  specs:
+  - gcc@12

recipes/linaro-forge/23.0.3/config.yaml

@@ -0,0 +1,6 @@
+name: linaro
+store: /user-tools
+spack:
+    repo: https://github.com/spack/spack.git
+    commit: releases/v0.21
+modules: True

recipes/linaro-forge/23.0.3/environments.yaml

@@ -0,0 +1,9 @@
+linaro:
+  views:
+    default:
+  compiler:
+      - toolchain: gcc
+        spec: gcc
+  unify: true
+  specs:
+  - linaro-forge +accept-eula

recipes/linaro-forge/23.0.3/modules.yaml

@@ -0,0 +1,23 @@
+modules:
+  # Paths to check when creating modules for all module sets
+  prefix_inspections:
+    bin:
+      - PATH
+    lib:
+      - LD_LIBRARY_PATH
+    lib64:
+      - LD_LIBRARY_PATH
+
+  default:
+    arch_folder: false
+    # Where to install modules
+    roots:
+      tcl: /user-environment/modules
+    tcl:
+      all:
+        autoload: none
+      hash_length: 0
+      exclude_implicits: true
+      exclude: ['%[email protected]', 'gcc %[email protected]']
+      projections:
+        all: '{name}/{version}'

recipes/linaro-forge/23.0.3/repo/packages/linaro-forge/package.py

@@ -0,0 +1,137 @@
+# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
+# Spack Project Developers. See the top-level COPYRIGHT file for details.
+#
+# SPDX-License-Identifier: (Apache-2.0 OR MIT)
+
+import os
+import platform
+import subprocess
+
+from spack.package import *
+
+
+class LinaroForge(Package):
+    """Build reliable and optimized code for the right results on multiple
+    Server and HPC architectures, from the latest compilers and C++ standards
+    to Intel, 64-bit Arm, AMD, OpenPOWER and Nvidia GPU hardware. Linaro Forge
+    combines Linaro DDT, the leading debugger for time-saving high performance
+    application debugging, Linaro MAP, the trusted performance profiler for
+    invaluable optimization advice across native and Python HPC codes, and
+    Linaro Performance Reports for advanced reporting capabilities."""
+
+    homepage = "https://www.linaroforge.com"
+    maintainers("kenche-linaro")
+
+    if platform.machine() in ["aarch64", "arm64"]:
+        version(
+            "23.0.3", sha256="a7e23ef2a187f8e2d6a6692cafb931c9bb614abf58e45ea9c2287191c4c44f02"
+        )
+        version(
+            "23.0.2", sha256="698fda8f7cc05a06909e5dcc50b9956f94135d7b12e84ffb21999a5b45c70c74"
+        )
+        version(
+            "23.0.1", sha256="552e4a3f408ed4eb5f1bfbb83c94530ee8733579c56c3e98050c0ad2d43eb433"
+        )
+        version("23.0", sha256="7ae20bb27d539751d1776d1e09a65dcce821fc6a75f924675439f791261783fb")
+        version(
+            "22.1.4", sha256="4e2af481a37b4c99dba0de6fac75ac945316955fc4170d06e321530adea7ac9f"
+        )
+        version(
+            "21.1.3", sha256="4a4ff7372aad5a31fc9e18b7b6c493691ab37d8d44a3158584e62d1ab82b0eeb"
+        )
+    elif platform.machine() == "ppc64le":
+        version(
+            "23.0.3", sha256="5ff9770f4bc4a2df4bac8a2544a9d6bad9fba2556420fa2e659e5c21e741caf7"
+        )
+        version(
+            "23.0.2", sha256="181b157bdfc8609b49addf63023f920ebb609dbc9a126e9dc26605188b756ff0"
+        )
+        version(
+            "23.0.1", sha256="08cffef2195ea96872d56e827f320eed40aaa82fd3b62d4c661a598fb2fb3a47"
+        )
+        version("23.0", sha256="0962c7e0da0f450cf6daffe1156e1f59e02c9f643df458ec8458527afcde5b4d")
+        version(
+            "22.1.3", sha256="6479c3a4ae6ce6648c37594eb0266161f06d9f89010fca9299855848661dda49"
+        )
+        version(
+            "22.0.4", sha256="f4cb5bcbaa67f9209299fe4653186a2829760b8b16a2883913aa43766375b04c"
+        )
+        version(
+            "21.1.3", sha256="eecbc5686d60994c5468b2d7cd37bebe5d9ac0ba37bd1f98fbfc69b071db541e"
+        )
+    elif platform.machine() == "x86_64":
+        version(
+            "23.0.3", sha256="f2a010b94838f174f057cd89d12d03a89ca946163536eab178dd1ec877cdc27f"
+        )
+        version(
+            "23.0.2", sha256="565f0c073c6c8cbb06c062ca414e3f6ff8c6ca6797b03d247b030a9fbc55a5b1"
+        )
+        version(
+            "23.0.1", sha256="1d681891c0c725363f0f45584c9b79e669d5c9782158453b7d24b4b865d72755"
+        )
+        version("23.0", sha256="f4ab12289c992dd07cb1a15dd985ef4713d1f9c0cf362ec5e9c995cca9b1cf81")
+        version(
+            "22.1.3", sha256="4f8a8b1df6ad712e89c82eedf4bd85b93b57b3c8d5b37d13480ff058fa8f4467"
+        )
+        version(
+            "22.0.4", sha256="a2c8c1da38b9684d7c4656a98b3fc42777b03fd474cd0bf969324804f47587e5"
+        )
+        version(
+            "21.1.3", sha256="03dc82f1d075deb6f08d1e3e6592dc9b630d406c08a1316d89c436b5874f3407"
+        )
+
+    variant(
+        "probe",
+        default=False,
+        description='Detect available PMU counters via "forge-probe" during install',
+    )
+
+    variant("accept-eula", default=False, description="Accept the EULA")
+
+    # forge-probe executes with "/usr/bin/env python"
+    depends_on("[email protected]:", type="build", when="+probe")
+
+    # Licensing
+    license_required = False
+
+    def url_for_version(self, version):
+        pre = "arm" if version < Version("23.0") else "linaro"
+        return f"https://downloads.linaroforge.com/{version}/{pre}-forge-{version}-linux-{platform.machine()}.tar"
+
+    @run_before("install")
+    def abort_without_eula_acceptance(self):
+        install_example = "spack install linaro-forge +accept-eula"
+        license_terms_path = os.path.join(self.stage.source_path, "license_terms")
+        if not self.spec.variants["accept-eula"].value:
+            raise InstallError(
+                "\n\n\nNOTE:\nUse +accept-eula "
+                + "during installation "
+                + "to accept the license terms in:\n"
+                + "  {0}\n".format(os.path.join(license_terms_path, "license_agreement.txt"))
+                + "  {0}\n\n".format(os.path.join(license_terms_path, "supplementary_terms.txt"))
+                + "Example: '{0}'\n".format(install_example)
+            )
+
+    def install(self, spec, prefix):
+        subprocess.call(["./textinstall.sh", "--accept-license", prefix])
+        if spec.satisfies("+probe"):
+            probe = join_path(prefix, "bin", "forge-probe")
+            subprocess.call([probe, "--install", "global"])
+
+    def setup_run_environment(self, env):
+        # Only PATH is needed for Forge.
+        # Adding lib to LD_LIBRARY_PATH can cause conflicts with Forge's internal libs.
+        env.clear()
+        env.prepend_path("PATH", join_path(self.prefix, "bin"))
+
+    @run_after("install")
+    def cscs_license(self):
+        cscs_license = join_path(self.prefix, "licences", "License")
+        with open(cscs_license, "w") as f:
+            # will expire end of May/2026
+            f.write("type=2\n")
+            f.write("serial_number=17741\n")
+            f.write("hostname=velan.cscs.ch\n")
+            f.write("serverport=4241\n")
+            f.write("features=ddt,map,perf-report,cuda,metrics-pack\n")
+            f.write("hash2=b013e17d168ebec7291c66401832d113963c0cb5\n")