Cannot import tensorflow first and then ROOT with 6.32 (BLAS version mismatch?)

[jonas-eschle]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

As described in root-project/root#15970, (reposting here), there is probably an issue with the BLAS version between ROOT and numpy/TF. I am not familiar with the build of ROOT/BLAS unfortunately.

It's installed with mamba create -n tmp311 root tensorflow python=3.11 and crashes on x86 architectures but works on macos.

Original Issue: (cc OP @ikrommyd)

Description

Importing tensorflow and then ROOT results in a segmentation fault with ROOT 6.32. The other way around works fine.

Reproducer

(root-bug) ➜  ~ python
Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
2024-07-02 14:04:03.356022: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-07-02 14:04:07.360190: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.

>>>
>>> import ROOT

 *** Break *** segmentation violation

ROOT version

6.32/0

Installation method

Conda: mamba create -n root-bug root=6.32 tensorflow

Operating system

Linux

Additional context

Full seg fault trace:

===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

Thread 10 (Thread 0x7f1039269640 (LWP 281196) "python"):
#0  0x00007f1095c8679a in __futex_abstimed_wait_common () from /lib64/libc.so.6
#1  0x00007f1095c88fa0 in pthread_cond_wait

MANY LINES OF CODE

[1] 281175 segmentation fault (core dumped) python

As suggested:

This is the same problem as described in #15309: if the BLAS version that is shipped by ROOT is different from the one that is shipped by other python packages, there will be a collision of different BLAS libraries.

For this case of using ROOT with Conda, the solution would be either to make sure the Conda build uses the same BLAS library as NumPy, using the BLA_VENDOR flag for CMake, of nothing else works set tmva-cpu=OFF for the Conda builds, as this is the component that makes ROOT vendor a BLAS library.

Installed packages

_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
_sysroot_linux-64_curr_repodata_hack 3                   haa98f57_10  
absl-py                   2.1.0           py311h06a4308_0  
afterimage                1.21              h28ea39c_1005    conda-forge
anyio                     4.2.0           py311h06a4308_0  
argon2-cffi               21.3.0             pyhd3eb1b0_0  
argon2-cffi-bindings      21.2.0          py311h5eee18b_0  
asttokens                 2.0.5              pyhd3eb1b0_0  
astunparse                1.6.3                      py_0  
async-lru                 2.0.4           py311h06a4308_0  
atk-1.0                   2.36.0               ha1a6a79_0  
attrs                     23.1.0          py311h06a4308_0  
babel                     2.11.0          py311h06a4308_0  
beautifulsoup4            4.12.3          py311h06a4308_0  
binutils                  2.40                 h4852527_7    conda-forge
binutils_impl_linux-64    2.40                 ha1999f0_7    conda-forge
binutils_linux-64         2.40                 hb3c18ed_9    conda-forge
blas                      1.0                    openblas  
bleach                    4.1.0              pyhd3eb1b0_0  
brotli-python             1.0.9           py311h6a678d5_8  
bzip2                     1.0.8                h5eee18b_6  
c-ares                    1.28.1               hd590300_0    conda-forge
c-compiler                1.7.0                hd590300_1    conda-forge
ca-certificates           2024.3.11            h06a4308_0  
cached-property           1.5.2                      py_0  
cairo                     1.18.0               hbb29018_2    conda-forge
certifi                   2024.6.2        py311h06a4308_0  
cffi                      1.16.0          py311h5eee18b_1  
cfitsio                   4.4.0                hbdc6101_1    conda-forge
charset-normalizer        2.0.4              pyhd3eb1b0_0  
comm                      0.2.1           py311h06a4308_0  
compilers                 1.7.0                ha770c72_1    conda-forge
cuda-crt-tools            12.4.131             h06a4308_0  
cuda-cudart               12.4.127             h99ab3db_0  
cuda-cudart_linux-64      12.4.127             hd681fbe_0  
cuda-nvcc-tools           12.4.131             h99ab3db_0  
cuda-nvrtc                12.4.127             h99ab3db_1  
cuda-nvtx                 12.4.127             h6a678d5_1  
cuda-nvvm-tools           12.4.131             h6a678d5_0  
cuda-version              12.4                 hbda6634_3  
cudnn                     8.9.7.29             h092f7fd_3    conda-forge
cxx-compiler              1.7.0                h00ab1b0_1    conda-forge
davix                     0.8.6                h5f3b820_0    conda-forge
debugpy                   1.6.7           py311h6a678d5_0  
decorator                 5.1.1              pyhd3eb1b0_0  
defusedxml                0.7.1              pyhd3eb1b0_0  
dm-tree                   0.1.7           py311h6a678d5_1  
entrypoints               0.4             py311h06a4308_0  
executing                 0.8.3              pyhd3eb1b0_0  
expat                     2.6.2                h6a678d5_0  
fftw                      3.3.10          nompi_hf1063bd_110    conda-forge
flatbuffers               24.3.25              h59595ed_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hd3eb1b0_0  
font-ttf-inconsolata      2.001                hcb22688_0  
font-ttf-source-code-pro  2.030                hd3eb1b0_0  
font-ttf-ubuntu           0.83                 h8b1ccd4_0  
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-anaconda            1                    h8fa9717_0  
fonts-conda-ecosystem     1                    hd3eb1b0_0  
fortran-compiler          1.7.0                heb67821_1    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h7b6447c_0  
ftgl                      2.4.0                hbcb1f35_0    conda-forge
gast                      0.5.3              pyhd3eb1b0_0  
gcc                       12.3.0              h915e2ae_13    conda-forge
gcc_impl_linux-64         12.3.0              h58ffeeb_13    conda-forge
gcc_linux-64              12.3.0               h9528a6a_9    conda-forge
gdk-pixbuf                2.42.12              hb9ae30d_0    conda-forge
gettext                   0.22.5               h59595ed_2    conda-forge
gettext-tools             0.22.5               h59595ed_2    conda-forge
gfortran                  12.3.0              h915e2ae_13    conda-forge
gfortran_impl_linux-64    12.3.0              h8f2110c_13    conda-forge
gfortran_linux-64         12.3.0               h5877db1_9    conda-forge
giflib                    5.2.2                hd590300_0    conda-forge
gl2ps                     1.4.2                hae5d5c5_1    conda-forge
glew                      2.1.0                h295c915_3  
glib                      2.80.2               h8a4344b_1    conda-forge
glib-tools                2.80.2               h73ef956_1    conda-forge
gobject-introspection     1.78.1          py311hd56c82f_0  
google-pasta              0.2.0              pyhd3eb1b0_0  
graphite2                 1.3.14               h295c915_1  
graphviz                  11.0.0               hc68bbd7_0    conda-forge
grpcio                    1.62.2          py311ha6695c7_0    conda-forge
gsl                       2.7.1                h6e86dc7_1  
gsoap                     2.8.123              h8dc497d_0    conda-forge
gtest                     1.14.0               hdb19cb5_1  
gtk2                      2.24.33              h73c1081_2  
gts                       0.7.6                hb67d8dd_3  
gxx                       12.3.0              h915e2ae_13    conda-forge
gxx_impl_linux-64         12.3.0              h2a574ab_13    conda-forge
gxx_linux-64              12.3.0               ha28b414_9    conda-forge
h5py                      3.11.0          nompi_py311h439e445_102    conda-forge
harfbuzz                  8.5.0                hfac3d4d_0    conda-forge
hdf5                      1.14.3          nompi_hdf9ad27_105    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
idna                      3.7             py311h06a4308_0  
ipykernel                 6.28.0          py311h06a4308_0  
ipyparallel               8.6.1           py311h06a4308_0  
ipython                   8.25.0          py311h06a4308_0  
jedi                      0.18.1          py311h06a4308_1  
jinja2                    3.1.4           py311h06a4308_0  
json5                     0.9.6              pyhd3eb1b0_0  
jsonschema                4.19.2          py311h06a4308_0  
jsonschema-specifications 2023.7.1        py311h06a4308_0  
jupyter-lsp               2.2.0           py311h06a4308_0  
jupyter_client            8.6.0           py311h06a4308_0  
jupyter_core              5.7.2           py311h06a4308_0  
jupyter_events            0.10.0          py311h06a4308_0  
jupyter_server            2.14.1          py311h06a4308_0  
jupyter_server_terminals  0.4.4           py311h06a4308_1  
jupyterlab                4.0.11          py311h06a4308_0  
jupyterlab_pygments       0.1.2                      py_0  
jupyterlab_server         2.25.1          py311h06a4308_0  
keras                     3.0.5           py311h06a4308_0  
kernel-headers_linux-64   3.10.0              h57e8cba_10  
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
ld_impl_linux-64          2.40                 hf3520f5_7    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20240116.2      cxx17_h6a678d5_0  
libaec                    1.1.3                h59595ed_0    conda-forge
libasprintf               0.22.5               h661eb56_2    conda-forge
libasprintf-devel         0.22.5               h661eb56_2    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libcublas                 12.4.5.8             h99ab3db_1  
libcufft                  11.2.1.3             h99ab3db_1  
libcurand                 10.3.5.147           h99ab3db_1  
libcurl                   8.8.0                hca28451_1    conda-forge
libcusolver               11.6.1.9             h99ab3db_1  
libcusparse               12.3.1.170           h99ab3db_1  
libcxx                    17.0.6               h434a139_3    conda-forge
libcxxabi                 17.0.6               ha770c72_3    conda-forge
libdeflate                1.20                 hd590300_0    conda-forge
libedit                   3.1.20230828         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libexpat                  2.6.2                h59595ed_0    conda-forge
libffi                    3.4.4                h6a678d5_1  
libgcc-devel_linux-64     12.3.0             h6b66f73_113    conda-forge
libgcc-ng                 14.1.0               h77fa898_0    conda-forge
libgd                     2.3.3                h119a65a_9    conda-forge
libgettextpo              0.22.5               h59595ed_2    conda-forge
libgettextpo-devel        0.22.5               h59595ed_2    conda-forge
libgfortran-ng            14.1.0               h69a702a_0    conda-forge
libgfortran5              14.1.0               hc5f4f2c_0    conda-forge
libglib                   2.80.2               h8a4344b_1    conda-forge
libglu                    9.0.0                hf484d3e_1  
libgomp                   14.1.0               h77fa898_0    conda-forge
libgrpc                   1.62.2               h15f2491_0    conda-forge
libhwloc                  2.11.0          default_h5622ce7_1000    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
libllvm14                 14.0.6               hcd5def8_4    conda-forge
libllvm16                 16.0.6               hb3ce162_3    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libnvjitlink              12.4.127             h99ab3db_1  
libopenblas               0.3.21               h043d6bf_0  
libpng                    1.6.43               h2797004_0    conda-forge
libprotobuf               4.25.3               h08a7969_0    conda-forge
libre2-11                 2023.09.01           h5a48ba9_2    conda-forge
librsvg                   2.58.1               hadf69e7_0    conda-forge
libsanitizer              12.3.0              hb8811af_13    conda-forge
libsodium                 1.0.18               h7b6447c_0  
libsqlite                 3.46.0               hde9e2c9_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-devel_linux-64  12.3.0             h6b66f73_113    conda-forge
libstdcxx-ng              14.1.0               hc0a3c3a_0    conda-forge
libtiff                   4.6.0                h1dd3fc0_3    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp                   1.4.0                h2c329e2_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.16                 hd590300_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.7               hc051c1a_1    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
llvmlite                  0.43.0          py311hbde99c3_0    conda-forge
lz4-c                     1.9.4                h6a678d5_1  
markdown                  3.4.1           py311h06a4308_0  
markdown-it-py            2.2.0           py311h06a4308_1  
markupsafe                2.1.3           py311h5eee18b_0  
matplotlib-inline         0.1.6           py311h06a4308_0  
mdurl                     0.1.0           py311h06a4308_0  
metakernel                0.27.5             pyhd3eb1b0_0  
mistune                   2.0.4           py311h06a4308_0  
ml_dtypes                 0.3.1           py311ha02d727_0  
namex                     0.0.7           py311h06a4308_0  
nbclient                  0.8.0           py311h06a4308_0  
nbconvert                 7.10.0          py311h06a4308_0  
nbformat                  5.9.2           py311h06a4308_0  
nccl                      2.22.3.1             hbc370b7_0    conda-forge
ncurses                   6.5                  h59595ed_0    conda-forge
nest-asyncio              1.6.0           py311h06a4308_0  
ninja                     1.10.2               h06a4308_5  
ninja-base                1.10.2               hd09550d_5  
nlohmann_json             3.11.3               h59595ed_0    conda-forge
notebook                  7.0.8           py311h06a4308_0  
notebook-shim             0.2.3           py311h06a4308_0  
numba                     0.60.0          py311h4bc866e_0    conda-forge
numpy                     1.26.4          py311h24aa872_0  
numpy-base                1.26.4          py311hbfb1bba_0  
openssl                   3.3.1                h4ab18f5_1    conda-forge
opt_einsum                3.3.0              pyhd3eb1b0_1  
overrides                 7.4.0           py311h06a4308_0  
packaging                 23.2            py311h06a4308_0  
pandocfilters             1.5.0              pyhd3eb1b0_0  
pango                     1.54.0               h84a9a3c_0    conda-forge
parso                     0.8.3              pyhd3eb1b0_0  
pcre                      8.45                 h295c915_0  
pcre2                     10.44                h0f59acf_0    conda-forge
pexpect                   4.8.0              pyhd3eb1b0_3  
pip                       24.0            py311h06a4308_0  
pixman                    0.43.2               h59595ed_0    conda-forge
platformdirs              3.10.0          py311h06a4308_0  
portalocker               2.3.0           py311h06a4308_1  
prometheus_client         0.14.1          py311h06a4308_0  
prompt-toolkit            3.0.43          py311h06a4308_0  
prompt_toolkit            3.0.43               hd3eb1b0_0  
protobuf                  4.25.3          py311h12ddb61_0  
psutil                    5.9.0           py311h5eee18b_0  
pthread-stubs             0.3                  h0ce48e5_1  
ptyprocess                0.7.0              pyhd3eb1b0_2  
pure_eval                 0.2.2              pyhd3eb1b0_0  
pycparser                 2.21               pyhd3eb1b0_0  
pygments                  2.15.1          py311h06a4308_1  
pysocks                   1.7.1           py311h06a4308_0  
pythia8                   8.310           py311hb755f60_0    conda-forge
python                    3.11.9          hb806964_0_cpython    conda-forge
python-dateutil           2.9.0post0      py311h06a4308_2  
python-fastjsonschema     2.16.2          py311h06a4308_0  
python-flatbuffers        24.3.25            pyh59ac667_0    conda-forge
python-json-logger        2.0.7           py311h06a4308_0  
python_abi                3.11                    4_cp311    conda-forge
pytz                      2024.1          py311h06a4308_0  
pyyaml                    6.0.1           py311h5eee18b_0  
pyzmq                     25.1.2          py311h6a678d5_0  
re2                       2023.09.01           h7f4b329_2    conda-forge
readline                  8.2                  h5eee18b_0  
referencing               0.30.2          py311h06a4308_0  
requests                  2.32.2          py311h06a4308_0  
rfc3339-validator         0.1.4           py311h06a4308_0  
rfc3986-validator         0.1.1           py311h06a4308_0  
rich                      13.3.5          py311h06a4308_0  
root                      6.32.0          py311h3958ca8_2    conda-forge
root_base                 6.32.0          py311h22294ec_2    conda-forge
rpds-py                   0.10.6          py311hb02cf49_0  
scitokens-cpp             1.0.2                haea88ab_0    conda-forge
send2trash                1.8.2           py311h06a4308_0  
setuptools                69.5.1          py311h06a4308_0  
six                       1.16.0             pyhd3eb1b0_1  
snappy                    1.2.1                ha2e4443_0    conda-forge
sniffio                   1.3.0           py311h06a4308_0  
soupsieve                 2.5             py311h06a4308_0  
stack_data                0.2.0              pyhd3eb1b0_0  
sysroot_linux-64          2.17                h57e8cba_10  
tbb                       2021.12.0            h434a139_2    conda-forge
tensorboard               2.16.2             pyhd8ed1ab_0    conda-forge
tensorboard-data-server   0.7.0           py311h52d8a92_0  
tensorflow                2.16.2          cuda120py311h51447cc_0    conda-forge
tensorflow-base           2.16.2          cuda120py311h8a6ca57_0    conda-forge
tensorflow-estimator      2.16.2          cuda120py311h0c188d0_0    conda-forge
termcolor                 2.1.0           py311h06a4308_0  
terminado                 0.17.1          py311h06a4308_0  
tinycss2                  1.2.1           py311h06a4308_0  
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tornado                   6.4.1           py311h5eee18b_0  
tqdm                      4.66.4          py311h92b7b1e_0  
traitlets                 5.14.3          py311h06a4308_0  
typing-extensions         4.11.0          py311h06a4308_0  
typing_extensions         4.11.0          py311h06a4308_0  
tzdata                    2024a                h04d1e81_0  
urllib3                   2.2.2           py311h06a4308_0  
vdt                       0.4.4                h59595ed_0    conda-forge
vector-classes            1.4.5                h297d8ca_0    conda-forge
wcwidth                   0.2.5              pyhd3eb1b0_0  
webencodings              0.5.1           py311h06a4308_1  
websocket-client          1.8.0           py311h06a4308_0  
werkzeug                  3.0.3           py311h06a4308_0  
wheel                     0.43.0          py311h06a4308_0  
wrapt                     1.14.1          py311h5eee18b_0  
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.9                hb711507_1    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxcursor           1.2.0                h0b41bf4_1    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxft               2.3.8                hf69aa0a_0    conda-forge
xorg-libxpm               3.5.17               hd590300_0    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-libxt                1.3.0                hd590300_1    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h27cfd23_1007  
xrootd                    5.6.9           py311hb5a8382_0    conda-forge
xxhash                    0.8.2                hd590300_0    conda-forge
xz                        5.4.6                h5eee18b_1  
yaml                      0.2.5                h7b6447c_0  
zeromq                    4.3.5                h6a678d5_0  
zlib                      1.3.1                h4ab18f5_1    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

Environment info

active environment : tmp311
 active env location : /home/jonas/mambaforge/envs/tmp311
            shell level : 1
       user config file : /home/jonas/.condarc
 populated config files : /home/jonas/mambaforge/.condarc
                          /home/jonas/.condarc
          conda version : 24.5.0
    conda-build version : not installed
         python version : 3.9.16.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=skylake
                          __conda=24.5.0=0
                          __glibc=2.35=0
                          __linux=6.5.0=0
                          __unix=0=0
       base environment : /home/jonas/mambaforge  (writable)
      conda av data dir : /home/jonas/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/jonas/mambaforge/pkgs
                          /home/jonas/.conda/pkgs
       envs directories : /home/jonas/mambaforge/envs
                          /home/jonas/.conda/envs
               platform : linux-64
             user-agent : conda/24.5.0 requests/2.31.0 CPython/3.9.16 Linux/6.5.0-41-generic ubuntu/22.04.4 glibc/2.35 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.8
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False