Install from source on HPC

[pavelToman]

Hello micro-sams!
I would like to install your micro-sam on HPC for UGent researchers.
We are using EasyBuild for handling installations and softwares.
Could you help me please with installing micro-sam from sources by CMake, make, pip install? I can not use conda/mamba for this.
First of all I can not find which nifty I should install as a dependency, is it this one? https://github.com/DerThorsten/nifty/tree/master

[constantinpape]

Hi @pavelToman!

Could you help me please with installing micro-sam from sources by CMake, make, pip install? I can not use conda/mamba for this.

Yes, happy to help! It may take a bit of time, but if you get an easybuild recipe we'd also be interested in having a reference to it :). And in general this should be possible, all dependencies are either installable by CMake or pip.

First of all I can not find which nifty I should install as a dependency, is it this one? https://github.com/DerThorsten/nifty/tree/master

Yes, that is the correct nifty. The other dependency that might be a bit tricky to find / install is vigra: https://github.com/ukoethe/vigra

[pavelToman]

For now the vigra and nifty seems fine. I was able to build them.
But there is a problem with z5py, could you hint me please?
I am hitting error during make step:

In file included from /scratch/gent/vo/001/gvo00117/easybuild/RHEL8/cascadelake-ampere-ib/software/xtensor/0.21.10-foss-2023a/include/xtensor/xstorhpp:23,
                         from /scratch/gent/vo/001/gvo00117/easybuild/RHEL8/cascadelake-ampere-ib/software/xtensor/0.21.10-foss-2023a/include/xtensor/xbuffdaptor.hpp:22,
                         from /scratch/gent/vo/001/gvo00117/easybuild/RHEL8/cascadelake-ampere-ib/software/xtensor/0.21.10-foss-2023a/include/xtensor-pythoarray.hpp:17,
                         from /tmp/vsc47063/easybuild/build/z5py/2.0.17/foss-2023a/z5-2.0.17/src/python/lib/z5py.cxx:8:
        /scratch/gent/vo/001/gvo00117/easybuild/RHEL8/cascadelake-ampere-ib/software/xtensor/0.21.10-foss-2023a/include/xtensor/xtensor_simd.hpp:65:18: errset_simd has not been declared in xsimd
           65 |     using xsimd::set_simd;
              |                  ^~~~~~~~
        /scratch/gent/vo/001/gvo00117/easybuild/RHEL8/cascadelake-ampere-ib/software/xtensor/0.21.10-foss-2023a/include/xtensor/xtensor_simd.hpp:66:18: errload_simd has not been declared in xsimd
           66 |     using xsimd::load_simd;
              |                  ^~~~~~~~~
        /scratch/gent/vo/001/gvo00117/easybuild/RHEL8/cascadelake-ampere-ib/software/xtensor/0.21.10-foss-2023a/include/xtensor/xtensor_simd.hpp:67:18:or: store_simd has not been declared in xsimd
           67 |     using xsimd::store_simd;

... and many more same errors about simd.
I have in dependencies:

• xtensor v0.21.10
• xtensor-python v0.24.1
• xsimd v8.0.5

I can see there is maybe a problem related to: constantinpape/z5#218
nifty v1.2.1 has in the environment file:

• xtensor>=0.21,<0.22
• xtensor-python>=0.24,<0.25

z5py v2.0.17 has in the environment file:

• xtensor >=0.24,<0.25
• xtensor-python >=0.26,<0.27
• xsimd >=8,<9

and for z5py v2.0.16:

• xtensor>=0.20,<0.21
• xtensor-python>=0.23,<0.24
• xsimd

It seems I can not fullfit both the nifty and z5py requirements for xtensor.
Any idea please, how to solve this?

[constantinpape]

Can you see if building both nifty and z5py with these dependencies works:

z5py v2.0.17 has in the environment file:

* xtensor >=0.24,<0.25

* xtensor-python >=0.26,<0.27

* xsimd >=8,<9

I have a local build of nifty that works in these settings, and I assume that the pin of xtensor in nifty is too conservative.
Let me know if this works. If not I will double check my build environment and share it here (but will only have time for this tomorrow.)

[pavelToman]

Yes it works! I update versions of xtensor and xtensor-python and it seems nifty has no problem with more recent xtensor versions :) Thank you!

[pavelToman]

@constantinpape thank you for your help!
It seems micro-sam can be installed by EasyBuild and is ready to be tested by researchers now.

• {vis}[foss/2023a] micro-sam v1.0.1, torch_em v0.7.1, affogato v0.3.3, ... easybuilders/easybuild-easyconfigs#20636

[constantinpape]

Thanks @pavelToman! I will also add a link to the recipe in the docs.