New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Paraview recipe #39

Open

biddisco wants to merge 51 commits into eth-cscs:main from biddisco:paraview

biddisco commented Aug 23, 2023

This PR build a minimal paraview using EGL on clariden.
It requires stackinator patch PR
eth-cscs/stackinator#129

biddisco force-pushed the paraview branch from 0af747a to 0688c04 Compare

August 23, 2023 09:03

biddisco force-pushed the paraview branch from 96e8c61 to 98362e9 Compare

September 5, 2023 07:51

jfavre commented Feb 7, 2024

installation on ALPS will require two additional items for run-time enabling of the NVIDIA IndeX libraries and license.

The lib package can be found in https://www.paraview.org/files/dependencies/nvidia-index-libs-5.12.0.20231121-linux-aarch64.tar.bz2. untar-ing the file can be done anywhere (presumably under the ParaView install tree), as long as the lib is found in LD_LIBRARY_PATH.
Access and installation of the license file (which is obtained by private request to NVIDIA using my email) is documented at https://confluence.cscs.ch/display/VCUE/NVIDIA+IndeX. Thus, the environment variable NVINDEX_PVPLUGIN_HOME shall be set where the file nvindex_config.xml gets installed.

biddisco force-pushed the paraview branch from 98362e9 to 41e09f6 Compare

February 26, 2024 22:01

RMeli added app paraview labels

biddisco force-pushed the paraview branch from 41e09f6 to 14406cb Compare

August 30, 2024 13:09

Author

biddisco commented Aug 30, 2024

The PR can now be considered for merging. We support

eiger: zen2 : 
  paraview with OSMESA offscreen headless rendering, 
  python, 
  CDI for icon support, 
  raycasting with TBB  
  VTK-m filters using TBB
  + lots lots more

daint: gh200 : 
  paraview with EGL headless accelerated rendering, 
  nvidia index plugin, 
  python, 
  CDI support for icon
  TBB and raycasting
  CUDA enabled VTK-m filters 
  + more

Member

simonpintarelli commented Aug 30, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Aug 30, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

1 similar comment

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

4 similar comments

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Author

biddisco commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Author

biddisco commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

biddisco force-pushed the paraview branch from f50b5e0 to f4e8ed0 Compare

November 20, 2024 08:51

Author

biddisco commented Nov 20, 2024

Please test this and consider it for deployment on our clusters

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

Author

biddisco commented Nov 20, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.1

Author

biddisco commented Nov 20, 2024

please note, paraview release has been bumped from 5.13.0 to 5.13.1

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=todi;uarch=gh200;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 25, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 25, 2024

cscs-ci run alps;system=todi;uarch=gh200;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 25, 2024

I've just tried to build it manually on eiger, there is an error compiling h5hut on zen2 too.

simonpintarelli self-requested a review

November 25, 2024 20:38

biddisco and others added 20 commits

January 10, 2025 08:04


          Cleanup scripts/paths after rebase, add pvserver setup file etc

38ff8df


          Fix TBB find problem by setting TBB_ROOT

f8b0fb5


          Add cdi from albestro/alby/paraview instesad of c2sm to avoid version…

d8d9608

… incompatibility


          Use jfrog for license. Fix cdi, adios, plugin builds and submit scripts

ab12bd5


          Fix paraview python view setup for numpy etc

191f1ed


          rename amd-zen2 to plain zen2, fix scripts paths, names, etc

fe31924


          Add cluster name to RC launch script and cleanup script comments etc

c195036


          delete old/unused c2sm libs

723e1ab


          Rename todi to daint

59f9ef3


          change directory structure

b843e37


          update config.yaml

a8e2ca0


          Fix build script paths after version added to recipe location

457a395


          Set spack commit to recent SHA and get paraview source from jfrog

5d2e0ae


          update paraview to 5.13.1, add adios2, h5py and conduit

e986e1f


          Clean up build-env for plugins, add h5py for pv-python

93c493a


          Remove h5hut from gh200 recipe

f29043e


          Remove ispc customizations and set spack develop tag

b0ede57


          fix for spack@develop

f53c3b5


          paraview: cleanup views in environment setup

7ed8d62


          paraview: update 5.13.1 -> 5.13.2

a3f261d

biddisco force-pushed the paraview branch from 9e9107a to a3f261d Compare

January 15, 2025 10:14

Author

biddisco commented Jan 15, 2025 •

edited

Loading

cscs-ci run alps;system=todi;uarch=gh200;uenv=paraview:5.13.2

Author

biddisco commented Jan 15, 2025

Help @simonpintarelli - CI seems to fail with

['#SBATCH --partition=normal', '#SBATCH --reservation=icon', '#SBATCH --time=180', '#SBATCH --account=csstaff']
2025-01-15 16:15:59.256924: Exception while waiting submitting job to queue. Exception=last request: 200 {'tasks': {'4342e4b82769ac9897fce85bc64e74e9': {'created_at': '2025-01-15T15:15:29', 'data': 'sbatch: error: Batch job submission failed: Requested reservation is invalid', 'description': 'Finished with errors

it is using a reservation for icon on machine todi - I don't know where this is set or how to change it.
where is the CI info stored?


          build on santis too

e593a1e

Author

biddisco commented Jan 16, 2025

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.2

1 similar comment

Member

simonpintarelli commented Jan 16, 2025

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.2

Author

biddisco commented Jan 21, 2025

Several users have requested a uenv for paraview. What do I need to do to get this accepted as a valid solution for users?


          pin cray-mpich version

26ac004

simonpintarelli approved these changes

View reviewed changes

Member

simonpintarelli left a comment

lgtm

recipes/paraview/5.13.0/zen2/config.yaml Outdated

+              name: prgenv-gnu
+              spack:
+                commit: jb-develop
+                repo: /capstor/scratch/cscs/biddisco/spack-eiger

Member

simonpintarelli Aug 30, 2024

This directory isn't readable for everyone, the CI is failing to clone when calling stack-config [..]

[daint][simonpi@daint-ln001 simonpi]$ ls -alhd /capstor/scratch/cscs/biddisco/spack-eiger
drwx------+ 11 biddisco csstaff 4.0K Aug 21 23:10 /capstor/scratch/cscs/biddisco/spack-eiger

Member

simonpintarelli Aug 30, 2024

Is there a particular reason to use a forked spack repo? If only package.py have been changed, those recipes should be copied to repo/packages instead and then spack upstream used in config.yaml

recipes/paraview/5.13.1/gh200/environments.yaml Outdated

+                # raytracing in VTK/ParaView
+                - [email protected] ~mpi +denoiser +volumes ~apps ~glm
+                - ispc@main

Member

simonpintarelli Nov 25, 2024

It would be better to use a commit id instead.

Spack supports the following syntax
ispc@git.<git ref>=main
(https://spack.readthedocs.io/en/latest/basic_usage.html#git-versions)

recipes/paraview/5.13.1/gh200/post-install Outdated

+              # remove /dev/shm paths from build-env that will not be available to the final user
+              GCCLOC=$($SPACK_CMD location -i gcc@12)
+              sed -i "s|/dev/shm/biddisco/spack/lib/spack/env/gcc|$GCCLOC/bin|g"  $BUILD_ENV

Member

simonpintarelli Nov 27, 2024

Is this causing the incorrect paths in activate.sh?

Member

simonpintarelli commented Jan 21, 2025

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels