Merge pull request #2908 from easybuilders/4.7.x

release EasyBuild v4.7.1

.github/workflows/linting.yml

@@ -1,6 +1,9 @@
 name: Static Analysis
 on: [push, pull_request]
 
+permissions:
+  contents: read # to fetch code (actions/checkout)
+
 concurrency:
   group: ${{format('{0}:{1}:{2}', github.repository, github.ref, github.workflow)}}
   cancel-in-progress: true

.github/workflows/unit_tests.yml

@@ -1,6 +1,9 @@
 name: easyblocks unit tests
 on: [push, pull_request]
 
+permissions:
+  contents: read # to fetch code (actions/checkout)
+
 concurrency:
   group: ${{format('{0}:{1}:{2}', github.repository, github.ref, github.workflow)}}
   cancel-in-progress: true

README.rst

@@ -4,7 +4,7 @@
 
 .. image:: https://github.com/easybuilders/easybuild-easyblocks/workflows/easyblocks%20unit%20tests/badge.svg?branch=develop
 
-`EasyBuild <https://easybuilders.github.io/easybuild>`_ is a software build
+`EasyBuild <https://easybuild.io>`_ is a software build
 and installation framework that allows you to manage (scientific) software
 on High Performance Computing (HPC) systems in an efficient way.
 
@@ -13,7 +13,7 @@ EasyBuild. Easyblocks are Python modules that implement the install procedure fo
 (group of) software package(s). Together with the EasyBuild framework,
 they allow to easily build and install supported software packages.
 
-The EasyBuild documentation is available at http://easybuild.readthedocs.org/.
+The EasyBuild documentation is available at http://docs.easybuild.io/.
 
 The easybuild-easyblocks source code is hosted on GitHub, along
 with an issue tracker for bug reports and feature requests, see

RELEASE_NOTES

@@ -5,6 +5,45 @@ These release notes can also be consulted at http://easybuild.readthedocs.org/en
 
 The latest version of easybuild-easyblocks provides 248 software-specific easyblocks and 39 generic easyblocks.
 
+v4.7.1 (March 20th 2023)
+------------------------
+
+update/bugfix release
+
+- minor enhancements and updates, including:
+  - fix TensorFlow easyblock for new versions of Bazel & TensorFlow (#2854)
+  - make NAMD easyblock aware of (pre)testopts (#2856)
+  - update MesonNinja easyblock for Meson >=0.64.0 (#2861)
+  - update scipy easyblock for scipy >= 1.9.0 to use meson/ninja  (#2862, #2903)
+  - modify logic in QScintilla easyblock to find the PyQt5 sipdir in more places (#2868)
+  - add `testinstall` custom easyconfig parameter to PythonPackage easyblock (#2872)
+  - use -x option for "go install" in GoPackage generic easyblock, to print commands as they are executed (#2878)
+  - allow disabling pybind11 tests with `runtest = False` (#2892)
+  - call parent post_install_step in EasyBuildMeta easyblock (so postinstallcmds are taken into account) (#2893)
+  - update and enhance Maple easyblock for recent versions (#2895)
+  - relax glob pattern to find Mathematica install script (#2896)
+  - implement CUDA support in the ELPA EasyBlock & fix CPP configure issue on newer ELPA versions (#2898)
+  - update Trilinos easyblock to allow disabling of building tests and forward deps + support Trilinos v13.x (#2900)
+  - enhance Python easyblock to create non-versioned symlink for python-config + check for bin/python and bin/python-config in sanity check (#2904)
+- various bug fixes, including:
+  - do not use -g77 option when installing NVHPC 22.9+ (#2819)
+  - check that sanity_check_module_loaded attribute exists before querying it in PythonPackage easyblock (#2865)
+  - fix $JULIA_DEPOT_PATH in installation of multiple JuliaPackage extensions (#2869)
+  - fix checking of CUDA/ROCR-Runtime dependencies for Clang to determine default build targets (#2873)
+  - show template values of exts_default_options in PythonBundle (#2874)
+  - fix missing initialization of CMakeMake in CMakePythonPackage (#2876)
+  - fix error when failing pip version check during PythonPackage sanity check (#2877)
+  - handle templating correctly in CMakeMake when playing with configopts (#2882)
+  - avoid crash in test step of PyTorch easyblock if runtest is not a command (#2883)
+  - fix check configure option in FlexiBLAS easyblock (#2886)
+  - use older `ncgen -H` for older netCDF (#2889)
+  - fix linking numexpr with Intel MKL's VML library for imkl >= 2021.x (#2897)
+- other changes:
+  - only give read permissions in GitHub Actions workflows (#2863)
+  - use start dir of extension to install R packages (#2867)
+  - fix website/docs links in README (#2870)
+  - add deprecation notice to RPackage extensions with relative paths in start_dir (#2879)
+
 
 v4.7.0 (January 9th 2023)
 -------------------------

easybuild/easyblocks/__init__.py

@@ -43,7 +43,7 @@
 # recent setuptools versions will *TRANSFORM* something like 'X.Y.Zdev' into 'X.Y.Z.dev0', with a warning like
 #   UserWarning: Normalizing '2.4.0dev' to '2.4.0.dev0'
 # This causes problems further up the dependency chain...
-VERSION = LooseVersion('4.7.0')
+VERSION = LooseVersion('4.7.1')
 UNKNOWN = 'UNKNOWN'
 
 

easybuild/easyblocks/c/clang.py

@@ -177,18 +177,21 @@ def __init__(self, *args, **kwargs):
         build_targets = self.cfg['build_targets']
         # define build_targets if not set
         if build_targets is None:
+            deps = [dep['name'].lower() for dep in self.cfg.dependencies()]
             arch = get_cpu_architecture()
             try:
                 default_targets = DEFAULT_TARGETS_MAP[arch][:]
                 # If CUDA is included as a dep, add NVPTX as a target
-                if get_software_root("CUDA"):
-                    default_targets += ["NVPTX"]
+                # There are (old) toolchains with CUDA as part of the toolchain
+                cuda_toolchain = hasattr(self.toolchain, 'COMPILER_CUDA_FAMILY')
+                if 'cuda' in deps or cuda_toolchain:
+                    default_targets += ['NVPTX']
                 # For AMDGPU support we need ROCR-Runtime and
                 # ROCT-Thunk-Interface, however, since ROCT is a dependency of
                 # ROCR we only check for the ROCR-Runtime here
                 # https://openmp.llvm.org/SupportAndFAQ.html#q-how-to-build-an-openmp-amdgpu-offload-capable-compiler
-                if get_software_root("ROCR-Runtime"):
-                    default_targets += ["AMDGPU"]
+                if 'rocr-runtime' in deps:
+                    default_targets += ['AMDGPU']
                 self.cfg['build_targets'] = build_targets = default_targets
                 self.log.debug("Using %s as default build targets for CPU/GPU architecture %s.", default_targets, arch)
             except KeyError:

easybuild/easyblocks/e/easybuildmeta.py

@@ -157,6 +157,8 @@ def install_step(self):
     def post_install_step(self):
         """Remove setuptools.pth file that hard includes a system-wide (site-packages) path, if it is there."""
 
+        super(EB_EasyBuildMeta, self).post_install_step()
+
         setuptools_pth = os.path.join(self.installdir, self.pylibdir, 'setuptools.pth')
         if os.path.exists(setuptools_pth):
             setuptools_pth_txt = read_file(setuptools_pth)

easybuild/easyblocks/e/elpa.py

@@ -30,17 +30,21 @@
 @author: Kenneth Hoste (Ghent University)
 """
 import os
+from distutils.version import LooseVersion
 
+import easybuild.tools.environment as env
 from easybuild.easyblocks.generic.configuremake import ConfigureMake
 from easybuild.framework.easyconfig import CUSTOM
+from easybuild.toolchains.compiler.gcc import TC_CONSTANT_GCC
+from easybuild.toolchains.compiler.inteliccifort import TC_CONSTANT_INTELCOMP
 from easybuild.tools.build_log import EasyBuildError
 from easybuild.tools.config import build_option
 from easybuild.tools.filetools import apply_regex_substitutions
+from easybuild.tools.modules import get_software_root
 from easybuild.tools.systemtools import get_cpu_features, get_shared_lib_ext
 from easybuild.tools.toolchain.compiler import OPTARCH_GENERIC
 from easybuild.tools.utilities import nub
 
-
 ELPA_CPU_FEATURE_FLAGS = ['avx', 'avx2', 'avx512f', 'vsx', 'sse4_2']
 
 
@@ -176,6 +180,51 @@ def run_all_steps(self, *args, **kwargs):
 
         return super(EB_ELPA, self).run_all_steps(*args, **kwargs)
 
+    def configure_step(self):
+        """Configure step for ELPA"""
+
+        # Add nvidia GPU support if requested
+        cuda_root = get_software_root('CUDA')
+        self.log.info("Got cuda root: %s", cuda_root)
+        if cuda_root:
+            self.cfg.update('configopts', '--enable-nvidia-gpu')
+            self.cfg.update('configopts', '--with-cuda-path=%s' % cuda_root)
+            self.cfg.update('configopts', '--with-cuda-sdk-path=%s' % cuda_root)
+
+            cuda_cc = build_option('cuda_compute_capabilities') or self.cfg['cuda_compute_capabilities']
+            if not cuda_cc:
+                raise EasyBuildError('List of CUDA compute capabilities must be specified, either via '
+                                     'cuda_compute_capabilities easyconfig parameter or via '
+                                     '--cuda-compute-capabilities')
+
+            # ELPA's --with-NVIDIA-GPU-compute-capability only accepts a single architecture
+            if len(cuda_cc) != 1:
+                raise EasyBuildError('ELPA currently only supports specifying one CUDA architecture when '
+                                     'building. You specified cuda-compute-capabilities: %s', cuda_cc)
+            cuda_cc = cuda_cc[0]
+            cuda_cc_string = cuda_cc.replace('.', '')
+            self.cfg.update('configopts', '--with-NVIDIA-GPU-compute-capability=sm_%s' % cuda_cc_string)
+            self.log.info("Enabling nvidia GPU support for compute capability: %s", cuda_cc_string)
+            # There is a dedicated kernel for sm80, but only from version 2021.11.001 onwards
+            if float(cuda_cc) >= 8.0 and LooseVersion(self.version) >= LooseVersion('2021.11.001'):
+                self.cfg.update('configopts', '--enable-nvidia-sm80-gpu')
+
+        # From v2022.05.001 onwards, the config complains if CPP is not set, resulting in non-zero exit of configure
+        # C preprocessor to use for given comp_fam
+        cpp_dict = {
+            TC_CONSTANT_GCC: 'cpp',
+            TC_CONSTANT_INTELCOMP: 'cpp',
+        }
+        comp_fam = self.toolchain.comp_family()
+        if comp_fam in cpp_dict:
+            env.setvar('CPP', cpp_dict[comp_fam])
+        else:
+            raise EasyBuildError('ELPA EasyBlock does not know which C preprocessor to use for the '
+                                 'current compiler family (%s). Please add the correct preprocessor '
+                                 'for this compiler family to cpp_dict in the ELPA EasyBlock', comp_fam)
+
+        super(EB_ELPA, self).configure_step()
+
     def patch_step(self, *args, **kwargs):
         """Patch manual_cpp script to avoid using hardcoded /usr/bin/python."""
         super(EB_ELPA, self).patch_step(*args, **kwargs)

easybuild/easyblocks/f/flexiblas.py

@@ -138,7 +138,7 @@ def configure_step(self):
         # to allow easyconfig to override specifies settings
         for key, value in sorted(configopts.items()):
             opt = '-D%s=' % key
-            if key not in self.cfg['configopts']:
+            if opt not in self.cfg['configopts']:
                 self.cfg.update('configopts', opt + "'%s'" % value)
 
         # specify compiler commands with absolute paths, to ensure that RPATH wrapper scripts are used

easybuild/easyblocks/generic/cmakemake.py

@@ -141,7 +141,10 @@ def build_type(self):
 
     def prepend_config_opts(self, config_opts):
         """Prepends configure options (-Dkey=value) to configopts ignoring those already set"""
-        cfg_configopts = self.cfg['configopts']
+        # need to disable template resolution or it will remain the same for all runs
+        with self.cfg.disable_templating():
+            cfg_configopts = self.cfg['configopts']
+
         # All options are of the form '-D<key>=<value>'
         new_opts = ' '.join('-D%s=%s' % (key, value) for key, value in config_opts.items()
                             if '-D%s=' % key not in cfg_configopts)

easybuild/easyblocks/generic/cmakepythonpackage.py

@@ -53,11 +53,6 @@ def extra_options(extra_vars=None):
         extra_vars['runtest'][0] = None
         return extra_vars
 
-    def __init__(self, *args, **kwargs):
-        """Initialize with PythonPackage."""
-        PythonPackage.__init__(self, *args, **kwargs)
-        self._lib_ext = None  # From CMakeMake.__init__
-
     def configure_step(self, *args, **kwargs):
         """Main configuration using cmake"""
 

easybuild/easyblocks/generic/gopackage.py

@@ -131,6 +131,9 @@ def install_step(self):
             self.cfg['preinstallopts'],
             'go',
             'install',
+            # print commands as they are executed,
+            # including downloading and installing of package deps as listed in the go.mod file
+            '-x',
             self.cfg['installopts'],
         ])
         run_cmd(cmd, log_all=True, log_ok=True, simple=True)

easybuild/easyblocks/generic/juliapackage.py

@@ -96,11 +96,14 @@ def install_step(self):
         """Install Julia package with Pkg"""
 
         # prepend installation directory to Julia DEPOT_PATH
+        # extensions in a bundle can share their DEPOT_PATH
         # see https://docs.julialang.org/en/v1/manual/environment-variables/#JULIA_DEPOT_PATH
-        depot_path = os.getenv('JULIA_DEPOT_PATH')
-        if depot_path is not None:
-            depot_path = ':'.join([self.installdir, depot_path])
-        env.setvar('JULIA_DEPOT_PATH', depot_path)
+        depot_path = os.getenv('JULIA_DEPOT_PATH', [])
+        if depot_path:
+            depot_path = depot_path.split(os.pathsep)
+        if self.installdir not in depot_path:
+            depot_path = os.pathsep.join([depot for depot in [self.installdir] + depot_path if depot])
+            env.setvar('JULIA_DEPOT_PATH', depot_path)
 
         # command sequence for Julia.Pkg
         julia_pkg_cmd = ['using Pkg']

easybuild/easyblocks/generic/mesonninja.py

@@ -28,12 +28,16 @@
 @author: Kenneth Hoste (Ghent University)
 """
 
+from distutils.version import LooseVersion
 from easybuild.framework.easyblock import EasyBlock
 from easybuild.framework.easyconfig import CUSTOM
 from easybuild.tools.build_log import EasyBuildError
 from easybuild.tools.filetools import change_dir, create_unused_dir, which
+from easybuild.tools.modules import get_software_version
 from easybuild.tools.run import run_cmd
 
+DEFAULT_CONFIGURE_CMD = 'meson'
+
 
 class MesonNinja(EasyBlock):
     """
@@ -45,6 +49,7 @@ def extra_options(extra_vars=None):
         """Define extra easyconfig parameters specific to MesonNinja."""
         extra_vars = EasyBlock.extra_options(extra_vars)
         extra_vars.update({
+            'configure_cmd': [DEFAULT_CONFIGURE_CMD, "Configure command to use", CUSTOM],
             'separate_build_dir': [True, "Perform build in a separate directory", CUSTOM],
         })
         return extra_vars
@@ -73,8 +78,16 @@ def configure_step(self, cmd_prefix=''):
         if no_Dlibdir and no_libdir:
             self.cfg.update('configopts', '-Dlibdir=lib')
 
-        cmd = "%(preconfigopts)s meson --prefix %(installdir)s %(configopts)s %(sourcedir)s" % {
+        configure_cmd = self.cfg.get('configure_cmd') or DEFAULT_CONFIGURE_CMD
+        # Meson >= 0.64.0 has a deprecatation warning for running `meson [options]`
+        # instead of `meson setup [options]`
+        if (LooseVersion(get_software_version('Meson')) >= LooseVersion('0.64.0') and
+                configure_cmd == DEFAULT_CONFIGURE_CMD):
+            configure_cmd += ' setup'
+
+        cmd = "%(preconfigopts)s %(configure_cmd)s --prefix %(installdir)s %(configopts)s %(sourcedir)s" % {
             'configopts': self.cfg['configopts'],
+            'configure_cmd': configure_cmd,
             'installdir': self.installdir,
             'preconfigopts': self.cfg['preconfigopts'],
             'sourcedir': self.start_dir,

easybuild/easyblocks/generic/pythonbundle.py

@@ -62,21 +62,17 @@ def __init__(self, *args, **kwargs):
         self.cfg['exts_filter'] = EXTS_FILTER_PYTHON_PACKAGES
 
         # need to disable templating to ensure that actual value for exts_default_options is updated...
-        prev_enable_templating = self.cfg.enable_templating
-        self.cfg.enable_templating = False
+        with self.cfg.disable_templating():
+            # set default options for extensions according to relevant top-level easyconfig parameters
+            pypkg_keys = PythonPackage.extra_options().keys()
+            for key in pypkg_keys:
+                if key not in self.cfg['exts_default_options']:
+                    self.cfg['exts_default_options'][key] = self.cfg[key]
 
-        # set default options for extensions according to relevant top-level easyconfig parameters
-        pypkg_keys = PythonPackage.extra_options().keys()
-        for key in pypkg_keys:
-            if key not in self.cfg['exts_default_options']:
-                self.cfg['exts_default_options'][key] = self.cfg[key]
+            self.cfg['exts_default_options']['download_dep_fail'] = True
+            self.log.info("Detection of downloaded extension dependencies is enabled")
 
-        self.cfg['exts_default_options']['download_dep_fail'] = True
-        self.log.info("Detection of downloaded extension dependencies is enabled")
-
-        self.cfg.enable_templating = prev_enable_templating
-
-        self.log.info("exts_default_options: %s", self.cfg['exts_default_options'])
+            self.log.info("exts_default_options: %s", self.cfg['exts_default_options'])
 
         self.pylibdir = None
         self.all_pylibdirs = []

easybuild/easyblocks/generic/pythonpackage.py

@@ -250,6 +250,7 @@ def extra_options(extra_vars=None):
             'sanity_pip_check': [False, "Run 'python -m pip check' to ensure all required Python packages are "
                                         "installed and check for any package with an invalid (0.0.0) version.", CUSTOM],
             'runtest': [True, "Run unit tests.", CUSTOM],  # overrides default
+            'testinstall': [False, "Install into temporary directory prior to running the tests.", CUSTOM],
             'unpack_sources': [None, "Unpack sources prior to build/install. Defaults to 'True' except for whl files",
                                CUSTOM],
             # A version of 0.0.0 is usually an error on installation unless the package does really not provide a
@@ -283,7 +284,7 @@ def __init__(self, *args, **kwargs):
         self.sitecfgfn = 'site.cfg'
         self.sitecfglibdir = None
         self.sitecfgincdir = None
-        self.testinstall = False
+        self.testinstall = self.cfg['testinstall']
         self.testcmd = None
         self.unpack_options = self.cfg['unpack_options']
 
@@ -793,7 +794,7 @@ def sanity_check_step(self, *args, **kwargs):
         # load module early ourselves rather than letting parent sanity_check_step method do so,
         # since custom actions taken below require that environment is set up properly already
         # (especially when using --sanity-check-only)
-        if not self.sanity_check_module_loaded:
+        if hasattr(self, 'sanity_check_module_loaded') and not self.sanity_check_module_loaded:
             extension = self.is_extension or kwargs.get('extension', False)
             extra_modules = kwargs.get('extra_modules', None)
             self.fake_mod_data = self.sanity_check_load_module(extension=extension, extra_modules=extra_modules)
@@ -916,8 +917,9 @@ def sanity_check_step(self, *args, **kwargs):
                     if pip_check_errors:
                         raise EasyBuildError('\n'.join(pip_check_errors))
                 else:
-                    raise EasyBuildError("pip >= 9.0.0 is required for running '%s', found %s", (pip_check_command,
-                                                                                                 pip_version))
+                    raise EasyBuildError("pip >= 9.0.0 is required for running '%s', found %s",
+                                         pip_check_command,
+                                         pip_version)
             else:
                 raise EasyBuildError("Failed to determine pip version!")
 

easybuild/easyblocks/generic/rpackage.py

@@ -134,10 +134,14 @@ def make_cmdline_cmd(self, prefix=None):
         else:
             prefix = ''
 
-        if self.start_dir:
-            loc = os.path.join(self.ext_dir or os.path.sep, self.start_dir)
-        else:
+        loc = self.start_dir
+        if loc is None:
             loc = self.ext_dir or self.ext_src
+        elif not os.path.isabs(loc):
+            # TODO: deprecated behaviour in framework 4.7.1, remove after 5.0
+            loc = os.path.join(self.ext_dir or os.path.sep, loc)
+            deprecation_msg = "Found relative path in start_dir, please upgrade to easybuild-framework>=4.7.1"
+            self.log.deprecated(deprecation_msg, '5.0')
 
         cmd = ' '.join([
             self.cfg['preinstallopts'],