Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDSC: PKG - expanse/0.17.3/gpu/b - Missing Amber GPU(example application) #50

Open
nwolter opened this issue Mar 22, 2023 · 14 comments
Open

Comments

@nwolter
Copy link

nwolter commented Mar 22, 2023

No description provided.

@mkandes
Copy link
Member

mkandes commented May 5, 2023

I still need to retry a build for amber22 on GPU.

@mkandes mkandes changed the title SDSC: PKG - expanse/0.17.3/gpu/a - Missing Amber GPU(example application) SDSC: PKG - expanse/0.17.3/gpu/b - Missing Amber GPU(example application) May 5, 2023
@mkandes
Copy link
Member

mkandes commented Jul 28, 2023

@mkandes - Jerry has an updated Amber Spack package that includes Python extensions with miniconda environment.

@mkandes
Copy link
Member

mkandes commented Aug 27, 2023

Build still failing ...

/home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-amber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/src/pmemd/src/cuda/gputypes.h:837:15: warning: 'status' may be used uninitialized in this function [-Wmaybe-uninitialized]
  837 |         printf("%s %s\n", s, cudaGetErrorString(status)); \
      |               ^
In file included from /home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-amber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/src/pmemd/src/cuda/base_gpuContext.h:5,
                 from /home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-amber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/src/pmemd/src/cuda/gpuContext.h:4,
                 from /home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-amber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/src/pmemd/src/cuda/gpu.cpp:24:
/home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-amber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/src/pmemd/src/cuda/gpuBuffer.h:220:15: note: 'status' was declared here
  220 |   cudaError_t status;
      |               ^~~~~~
[ 41%] Linking CXX static library libpmemd_cuda_DPFP.a
[ 41%] Built target pmemd_cuda_DPFP
make: *** [Makefile:156: all] Error 2
==> Error: ProcessError: Command exited with status 2:
    'make' '-j10' 'install'

56 errors found in build log:
     13752           |
     13753     Warning: 'sc_eel' may be used uninitialized in this function [-W
               maybe-uninitialized]
     13754     [ 72%] Building Fortran object AmberTools/src/quick/src/CMakeFil
               es/libquick.dir/dlfind/dlf_time.f90.o
     13755     [ 72%] Building Fortran object src/pmemd/src/CMakeFiles/pmemd.di
               r/gti.F90.o
     13756     [ 72%] Building Fortran object AmberTools/src/quick/src/CMakeFil
               es/libquick.dir/modules/quick_method_module.f90.o
     13757     [ 72%] Building Fortran object AmberTools/src/quick/src/CMakeFil
               es/libquick.dir/subs/zmake.f90.o
  >> 13758     /home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-am
               ber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/build/CMakeFil
               es/miniconda/install/lib/python3.11/site-packages/setuptools/dis
               t.py:755: SetuptoolsDeprecationWarning: Invalid dash-separated o
               ptions
     13759     !!
     ...
     128727    /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-cascad
               elake/gcc-10.2.0/cuda-11.2.2-blza2psofa3wr2zumqrnh4je2f7ze3mx/in
               clude/thrust/detail/device_delete.inl(42): here
     128728                instantiation of "void thrust::device_delete(thrust:
               :device_ptr<T>, size_t) [with T=xray::Sym33]"
     128729    /home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-am
               ber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/src/pmemd/src/
               xray/cuda/src/xray/BulkMaskGPU.cu(116): here
     128730    
  >> 128731    /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-cascad
               elake/gcc-10.2.0/cuda-11.2.2-blza2psofa3wr2zumqrnh4je2f7ze3mx/in
               clude/thrust/detail/type_traits/pointer_traits.h(179): error: cl
               ass "thrust::detail::pointer_difference<<error-type>>" has no me
               mber "type"
     128732              detected during:
     128733                instantiation of class "thrust::detail::pointer_trai
               ts<Ptr> [with Ptr=<error-type>]"
               ...
>> 128975    CMake Error at pmemd_xray_cuda_generated_BulkMaskGPU.cu.o.RELEAS
               E.cmake:278 (message):
     128976      Error generating file
     128977      /home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-
               amber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-src/build/src/pm
               emd/src/xray/cuda/CMakeFiles/pmemd_xray_cuda.dir/src/xray/./pmem
               d_xray_cuda_generated_BulkMaskGPU.cu.o
     128978    
     128979    
  >> 128980    make[2]: *** [src/pmemd/src/xray/cuda/CMakeFiles/pmemd_xray_cuda
               .dir/build.make:77: src/pmemd/src/xray/cuda/CMakeFiles/pmemd_xra
               y_cuda.dir/src/xray/pmemd_xray_cuda_generated_BulkMaskGPU.cu.o] 
               Error 1
  >> 128981    make[1]: *** [CMakeFiles/Makefile2:7326: src/pmemd/src/xray/cuda
               /CMakeFiles/pmemd_xray_cuda.dir/all] Error 2
     128982    make[1]: *** Waiting for unfinished jobs....
     128983    [ 29%] Building NVCC (Device) object src/pmemd/src/cuda/CMakeFil
               es/pmemd_cuda_DPFP.dir/pmemd_cuda_DPFP_generated_kNeighborList.c
               u.o
     128984    [ 29%] Building CXX object AmberTools/src/cpptraj/src/CMakeFiles
               /cpptraj_common_obj_cuda.dir/ByteRoutines.cpp.o
     128985    [ 29%] Building CXX object AmberTools/src/cpptraj/src/CMakeFiles
               /cpptraj_common_obj_cuda.dir/CharMask.cpp.o
     128986    In file included from /cm/shared/apps/spack/0.17.3/gpu/b/opt/spa
               ck/linux-rocky8-cascadelake/gcc-10.2.0/cuda-11.2.2-blza2psofa3wr
               2zumqrnh4je2f7ze3mx/include/cub/util_arch.cuh:36,
     128987                     from /cm/shared/apps/spack/0.17.3/gpu/b/opt/spa
               ck/linux-rocky8-cascadelake/gcc-10.2.0/cuda-11.2.2-blza2psofa3wr
               2zumqrnh4je2f7ze3mx/include/cub/config.cuh:35,

See build log for details:
  /home/mkandes/.spack/0.17.3/gpu/b/var/spack/stage/spack-stage-amber-22-ff5fb77t7urxsztxciszke66xfifyueh/spack-build-out.txt

==> Error: Terminating after first install failure: ProcessError: Command exited with status 2:
    'make' '-j10' 'install'
real 23393.62
user 5415.09
sys 1681.28
ERROR: spack install failed.

mkandes added a commit that referenced this issue Sep 20, 2023
We've encounted persistent build issues when utilizing CUDA 11.2.2 to
build Amber 22 with GPU support using Spack on Expanse [1]. To get around
these issues, we're deploying this compiler and package dependency chain
into the expanse/0.17.3/gpu/b instance to support builds with CUDA
10.2.89. We expect reverting to the older compiler and CUDA versions
may resolve the issue with Amber 22 as the deployment on TSCC2 required
such a fix as well [2].

[1] #50

[2] [mkandes@login1 ~]$ module spider amber/22-4zxr2dr

----------------------------------------------------------------------------
  amber: amber/22-4zxr2dr
----------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "amber/22-4zxr2dr" module is available to load.

      shared  gpu/0.17.3  gcc/8.5.0-mf5bqu2  intel-mpi/2019.10.317-f7l5rk4

    Help:
      Amber is a suite of biomolecular simulation programs. Note: A manual
      download is required for Amber. Spack will search your current directory
      for the download file. Alternatively, add this file to a mirror so that
      Spack can find it. For instructions on how to set up a mirror, see
      http://spack.readthedocs.io/en/latest/mirrors.html

[mkandes@login1 ~]$
@mkandes
Copy link
Member

mkandes commented Sep 20, 2023

Amber22 with GPU-accelerated support has consistently failed to build successfully on Expanse with CUDA 11.2.2. As such, we've decided to deploy an older gcc 8.x compiler and package dependency chain to support CUDA 10.2.89 within the expanse/0.17.3.gpu/b instance.

440b8dd

@mkandes
Copy link
Member

mkandes commented Sep 21, 2023

Build succeeded.

[spack_gpu@login01 amber]$ pwd
/home/spack_gpu/software/amber
[spack_gpu@login01 amber]$ ls -lahtr
total 1.7G
-rw-r----- 1 spack_gpu spack   99M Feb  8  2023 Amber20.tar.bz2
-rw-r----- 1 spack_gpu spack  103M Feb  8  2023 Amber22.tar.bz2
-rw-r----- 1 spack_gpu spack  461M Feb  8  2023 AmberTools20.tar.bz2
-rw-r----- 1 spack_gpu spack  495M Feb  8  2023 AmberTools22.tar.bz2
drwxr-xr-x 8 spack_gpu spack     8 Apr 24 13:14 ..
-rw-r--r-- 1 spack_gpu spack  131K Apr 24 13:38 [email protected]
-rw-r--r-- 1 spack_gpu spack   68K May  5 12:58 [email protected]
-rw-r--r-- 1 spack_gpu spack  4.5K May  6 09:14 [email protected]
-rw-r--r-- 1 spack_gpu spack  544M Aug 27 12:25 AmberTools23.tar.bz2
-rw-r--r-- 1 spack_gpu spack   46K Aug 27 13:22 [email protected]
-rw-r--r-- 1 spack_gpu spack   47K Aug 27 15:09 [email protected]
-rw-r--r-- 1 spack_gpu use300 4.0K Sep 20 18:13 [email protected]
drwxr-xr-x 2 spack_gpu spack    14 Sep 20 18:14 .
-rw-r--r-- 1 spack_gpu spack   24M Sep 20 18:51 [email protected]
[spack_gpu@login01 amber]$ tail -n 20 [email protected] 
-- Installing: /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/quick.cuda.MPI
-- Set runtime path of "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/quick.cuda.MPI" to "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc//lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/openmpi-4.1.3-v2ei3gevopnyr3jdn5bxhz4rvysipbdy/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/cuda-10.2.89-yx5cxnumts5as2ldjgtbd7qph32shgdp/lib64"
-- Installing: /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/test-api.cuda.MPI
-- Set runtime path of "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/test-api.cuda.MPI" to "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc//lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/openmpi-4.1.3-v2ei3gevopnyr3jdn5bxhz4rvysipbdy/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/cuda-10.2.89-yx5cxnumts5as2ldjgtbd7qph32shgdp/lib64"
-- Installing: /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/gpuP2PCheck
-- Set runtime path of "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/gpuP2PCheck" to "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc//lib"
-- Installing: /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_SPFP
-- Set runtime path of "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_SPFP" to "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc//lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-fortran-4.5.3-74le6xkzq7y5mfyk7uomiec6vle2gyh3/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/cuda-10.2.89-yx5cxnumts5as2ldjgtbd7qph32shgdp/lib64:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-c-4.8.1-4j6gp7oeeuj265rypnz7nmwpubenlrkm/lib"
-- Installing: /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_SPFP.MPI
-- Set runtime path of "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_SPFP.MPI" to "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc//lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-fortran-4.5.3-74le6xkzq7y5mfyk7uomiec6vle2gyh3/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/openmpi-4.1.3-v2ei3gevopnyr3jdn5bxhz4rvysipbdy/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/cuda-10.2.89-yx5cxnumts5as2ldjgtbd7qph32shgdp/lib64:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-c-4.8.1-4j6gp7oeeuj265rypnz7nmwpubenlrkm/lib"
-- Installing: /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_DPFP
-- Set runtime path of "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_DPFP" to "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc//lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-fortran-4.5.3-74le6xkzq7y5mfyk7uomiec6vle2gyh3/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/cuda-10.2.89-yx5cxnumts5as2ldjgtbd7qph32shgdp/lib64:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-c-4.8.1-4j6gp7oeeuj265rypnz7nmwpubenlrkm/lib"
-- Installing: /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_DPFP.MPI
-- Set runtime path of "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc/bin/pmemd.cuda_DPFP.MPI" to "/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc//lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-fortran-4.5.3-74le6xkzq7y5mfyk7uomiec6vle2gyh3/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/openmpi-4.1.3-v2ei3gevopnyr3jdn5bxhz4rvysipbdy/lib:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/cuda-10.2.89-yx5cxnumts5as2ldjgtbd7qph32shgdp/lib64:/cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/netcdf-c-4.8.1-4j6gp7oeeuj265rypnz7nmwpubenlrkm/lib"
==> amber: Successfully installed amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc
  Fetch: 7.38s.  Build: 36m 40.48s.  Total: 36m 47.87s.
[+] /cm/shared/apps/spack/0.17.3/gpu/b/opt/spack/linux-rocky8-skylake_avx512/gcc-8.4.0/amber-22-ulauqq7xr3cxbwiexgwxljgfqba6nonc
real 2248.27
user 10547.80
sys 2293.10
[spack_gpu@login01 amber]$

@mkandes
Copy link
Member

mkandes commented Sep 21, 2023

[mkandes@login02 ~]$ module spider amber/22/ulauqq7-omp

----------------------------------------------------------------------------
  amber/22: amber/22/ulauqq7-omp
----------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "amber/22/ulauqq7-omp" module is available to load.

      gpu/0.17.3b  gcc/8.4.0/xiuwkua  openmpi/4.1.3/v2ei3ge
 
    Help:
      Amber is a suite of biomolecular simulation programs. Note: A manual
      download is required for Amber. Spack will search your current directory
      for the download file. Alternatively, add this file to a mirror so that
      Spack can find it. For instructions on how to set up a mirror, see
      http://spack.readthedocs.io/en/latest/mirrors.html
      


 

[mkandes@login02 ~]$

@mkandes
Copy link
Member

mkandes commented Sep 21, 2023

@nwolter - Ready for testing.

@nwolter
Copy link
Author

nwolter commented Sep 22, 2023

@mkandes I assume you ran this as well? I am getting an error.
$ cat amber.version22.25332963.err
Lmod has detected the following error: These module(s) or extension(s) exist
but cannot be loaded as requested: "amber"
Try: "module spider amber" to see how to load the module(s).

Currently Loaded Modules:

  1. slurm/expanse/21.08.8 4) cuda/10.2.89/yx5cxnu
  2. gpu/0.17.3b (g) 5) ucx/1.10.1/fkguskk
  3. gcc/8.4.0/xiuwkua 6) openmpi/4.1.3/v2ei3ge

Where:
g: built natively for Intel Skylake

/cm/local/apps/slurm/var/spool/job25332963/slurm_script: line 21: pmemd.cuda: command not found

@nwolter
Copy link
Author

nwolter commented Sep 25, 2023

Need to load amber/22 specifically. Example script can be found at:
/expanse/lustre/projects/use300/nickel/2023/amber/gpu/OneGPU.sb

@mkandes
Copy link
Member

mkandes commented Sep 25, 2023

Build spec and standard output committed back to deployment branch.

ca6b660

@mkandes
Copy link
Member

mkandes commented Sep 26, 2023

amber@22 for expanse/0.17.3/cpu/b would no longer build with the updated Spack package used to deploy amber@22 in expanse/0.17.3/gpu/b.

47 errors found in build log:
     15827    [ 74%] Building CXX object AmberTools/src/packmol_memgen/packmol_
              memgen/lib/memembed/CMakeFiles/memembed.dir/src/main.cpp.o
     15828    [ 74%] Building CXX object AmberTools/src/moft/CMakeFiles/metatwi
              st.dir/metatwist.cpp.o
     15829    [ 74%] Building CXX object AmberTools/src/packmol_memgen/packmol_
              memgen/lib/memembed/CMakeFiles/memembed.dir/src/direct.cpp.o
     15830    [ 74%] Linking CXX shared library libcpptraj.so
     15831    [ 74%] Built target libcpptraj
     15832    [ 74%] Building pytraj native library
  >> 15833    /scratch/spack_cpu/job_25389680/spack-stage/spack-stage-amber-22-
              xyfdgc34k6q3swpiiya26xy5u627cy5g/spack-src/build/CMakeFiles/minic
              onda/install/lib/python3.11/site-packages/setuptools/dist.py:755:
               SetuptoolsDeprecationWarning: Invalid dash-separated options
     15834    !!
     15835    
     15836            *********************************************************
              ***********************
     15837            Usage of dash-separated 'description-file' will not be su
              pported in future
     15838            versions. Please use the underscore name 'description_fil
              e' instead.
     15839    

     ...

     16180          |         1
     16181    Warning: Missing actual argument for argument '_formal_0' at (1)
     16182    [ 75%] Building Fortran object AmberTools/src/quick/src/CMakeFile
              s/libquick.dir/dlfind/dlf_hdlc_primitive.f90.o
     16183    [ 75%] Building Fortran object AmberTools/src/quick/src/CMakeFile
              s/libquick.dir/modules/quick_lri_module.f90.o
     16184    [ 75%] Building Fortran object AmberTools/src/quick/src/CMakeFile
              s/libquick.dir/modules/quick_all_module.f90.o
     16185    [ 75%] Building Fortran object AmberTools/src/quick/src/CMakeFile
              s/libquick.dir/modules/quick_molden_module.f90.o
  >> 16186    pytraj/analysis/c_action/actionlist.cpp:23023:1: error: too many 
              initializers for 'PyTypeObject' {aka '_typeobject'}
     16187    23023 | };
     16188          | ^
  >> 16189    pytraj/analysis/c_action/actionlist.cpp:23164:1: error: too many 
              initializers for 'PyTypeObject' {aka '_typeobject'}
     16190    23164 | };
     16191

@mkandes
Copy link
Member

mkandes commented Sep 26, 2023

The issue may have been that the python version installed by the latest miniconda installer was no longer compatible with the AmberTools23.

>> 16906    pytraj/analysis/c_action/actionlist.cpp:212:12: fatal error: long
              intrepr.h: No such file or directory
     16907      212 |   #include "longintrepr.h"
     16908          |            ^~~~~~~~~~~~~~~
     16909    compilation terminated.
  >> 16910    error: command '/cm/shared/apps/spack/0.17.3/cpu/b/lib/spack/env/
              gcc/gcc' failed with exit code 1
  >> 16911    make[2]: *** [AmberTools/src/pytraj/CMakeFiles/pytraj.dir/build.m
              ake:533: AmberTools/src/pytraj/CMakeFiles/python-build/pytraj-bui
              ld.stamp] Error 1
  >> 16912    make[1]: *** [CMakeFiles/Makefile2:6529: AmberTools/src/pytraj/CM
              akeFiles/pytraj.dir/all] Error 2
     16913    make[1]: *** Waiting for unfinished jobs....
     16914    Scanning dependencies of target sander.LES
     16915    Scanning dependencies of target libsander
     16916    Scanning dependencies of target libsanderles
     16917    [ 84%] Building Fortran object AmberTools/src/sander/CMakeFiles/s
              ander.LES.dir/memory_module.F90.o
     16918    [ 84%] Building Fortran object AmberTools/src/sander/CMakeFiles/s
              ander.LES.dir/state.F90.o

See build log for details:
  /scratch/spack_cpu/job_25390212/spack-stage/spack-stage-amber-22-xyfdgc34k6q3swpiiya26xy5u627cy5g/spack-build-out.txt

==> Error: Terminating after first install failure: ProcessError: Command exited with status 2:
    'make' '-j16' 'install'
real 370.23
user 1412.86
sys 1533.42
slurmstepd: error: *** JOB 25390212 ON exp-15-56 CANCELLED AT 2023-09-26T12:46:45 ***

Note, however, this is not entirely clear if this is the problem.

@mkandes
Copy link
Member

mkandes commented Sep 26, 2023

Attempting to restart from the spack/spack upstream was not an option either as they have still not implemented the cmake-based build process for amber@22.

# ==> amber: Executing phase: 'install'
# ==> [2023-09-26-13:25:10.181829] Installing ambertools_tmpdir to .
# ==> [2023-09-26-13:25:17.458037] FILTER FILE: AmberTools/src/configure2 [replacing "\-x\ \/bin\/csh"]
# ==> [2023-09-26-13:25:17.465136] './configure' '--skip-python' '--with-netcdf' '/cm/shared/apps/spack/0.17.3/cpu/b/opt/spack/linux-rocky8-zen2/gcc-10.2.0/netcdf-fortran-4.5.3-i35rncs7ftem7fd2asrefkhbnc572be5' '-noX11' '--no-updates' 'gnu'
#   The old 'configure' approach to building Amber is no longer supported.
#   Please cd to the build folder, and look at run_cmake.
#
#   (If you really need the old behavior, you can comment out the exit
#   statment at the top of this file, but don't expect everything to work,
#   and you will probably need to make further changes.)

As such, we now have at least 3 or more separate amber packages required to support all of the versions currently deployed on SDSC systems. We need to reconcile them into a general Spack package and submit these changes upstream to the Spack project.

mkandes added a commit that referenced this issue Oct 3, 2023
After successfully using one of the latest amber Spack packages
developed by Jerry Greenberg here at SDSC to deploy a GPU-accelerated
version of Amber22 and AmberTool23 into expanse/0.17.3/gpu/b, it was
found this package was incapable of redeploying the same CPU-only
version into expanse/0.17.3/cpu/b. As such, we've had to revert to the
original configuration of the package used to deploy into
expanse/0.17.3/cpu/b. We'll eventaully need to reconcile these issues as
neither of these packages support older versions of Amber either. See
[1] for more information.

[1] #50
@mkandes
Copy link
Member

mkandes commented Oct 3, 2023

Successfully redeployed and tested amber@22 for expanse/0.17.3/cpu/b.

c9225f3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants