Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: could not load cppyy_backend library, details: #4

Open
amal-khailtash opened this issue Mar 17, 2021 · 11 comments
Open

Comments

@amal-khailtash
Copy link

amal-khailtash commented Mar 17, 2021

This is probably not PySysC specific issue but my cppyy installation. I followed the procedure to build PySysC and I get this error when cppyy is imported (when I run any of the python files in tests directory):

Traceback (most recent call last):
  File ".../PySysC/tests/modules.py", line 9, in <module>
    import cppyy
  File ".../PySysC/.venv/lib/python3.9/site-packages/cppyy/__init__.py", line 74, in <module>
    from ._cpython_cppyy import *
  File ".../PySysC/.venv/lib/python3.9/site-packages/cppyy/_cpython_cppyy.py", line 20, in <module>
    c = loader.load_cpp_backend()
  File ".../PySysC/.venv/lib/python3.9/site-packages/cppyy_backend/loader.py", line 85, in load_cpp_backend
    raise RuntimeError("could not load cppyy_backend library, details:\n%s" %
RuntimeError: could not load cppyy_backend library, details:
  libcppyy_backend.so: cannot open shared object file: No such file or directory
  .../.venv/lib/python3.9/site-packages/cppyy_backend/lib/libcppyy_backend.so: undefined symbol: _ZN11CppyyLegacy10TClassEdit9ShortTypeEPKci
  libcppyy_backend.cpython-39-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory
  .../.venv/lib/python3.9/site-packages/cppyy_backend/lib/libcppyy_backend.cpython-39-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory

Any ideas?

@wlav
Copy link

wlav commented Mar 17, 2021

Is there any libcppyy_backend*.so file in lib/python3.9/site-packages/cppyy_backend/lib?

@amal-khailtash
Copy link
Author

amal-khailtash commented Mar 18, 2021

Yes!

$ ll .venv/lib/python3.9/site-packages/cppyy_backend/lib/
total 68956
-rwxr-xr-x 1 akhailta asic_100g 62646768 Mar 17 17:23 libCling.so*
-rw-r--r-- 1 akhailta asic_100g     9722 Mar 17 17:23 libCoreLegacy.rootmap
-rwxr-xr-x 1 akhailta asic_100g  3468280 Mar 17 17:23 libCoreLegacy.so*
-rwxr-xr-x 1 akhailta asic_100g   170952 Mar 17 17:23 libcppyy_backend.so*   <<<<<<<
-rw-r--r-- 1 akhailta asic_100g     1386 Mar 17 17:23 libRIOLegacy.rootmap
-rwxr-xr-x 1 akhailta asic_100g  3603224 Mar 17 17:23 libRIOLegacy.so*
-rw-r--r-- 1 akhailta asic_100g      700 Mar 17 17:23 libThreadLegacy.rootmap
-rwxr-xr-x 1 akhailta asic_100g   402952 Mar 17 17:23 libThreadLegacy.so*

@amal-khailtash
Copy link
Author

amal-khailtash commented Mar 18, 2021

This symbol seems to point to the cppyy library itself:

$ echo _ZN11CppyyLegacy10TClassEdit9ShortTypeEPKci | demangle
CppyyLegacy::TClassEdit::ShortType(char const*, int)
CppyyLegacy::TClassEdit::ShortType(char const*, int)

The symbol is undefined in this library:

$ nm --dynamic --defined-only --extern-only .venv/lib/python3.9/site-packages/cppyy_backend/lib/libcppyy_backend.so | grep _ZN11CppyyLegacy10TClassEdit9ShortTypeEPKci
                 U _ZN11CppyyLegacy10TClassEdit9ShortTypeEPKci

As it is confirmed here:

$ nm --dynamic --defined-only --extern-only .venv/lib/python3.9/site-packages/cppyy_backend/lib/libcppyy_backend.so | grep _ZN11CppyyLegacy10TClassEdit9ShortTypeEPKci 

Not sure which library this symbol should be defined.

@wlav
Copy link

wlav commented Mar 18, 2021

CppyyLegacy::TClassEdit::ShortType(char const*, int) is defined in libCoreLegacy.so, but the mangled symbol should be _ZN11CppyyLegacy10TClassEdit9ShortTypeB5cxx11EPKci, not _ZN11CppyyLegacy10TClassEdit9ShortTypeEPKci. (I.e. with abi:cxx11.)

Was the gcc version with which libcppyy_backend.so was build older than 5? (You can use cppyy with gcc4.8 or later, but you'd have to install with the --no-binary=cppyy-cling option given to pip. That'll take a long time to build, though.)

@amal-khailtash
Copy link
Author

Yes, I was building with my system default version 4.8.5. I switched to gcc/8.3.0 and I can get passed that error. According to README.md, I am also supposed to build PySysC-SC, which I got from: https://git.minres.com/SystemC/PySysC-SC, bu I am having trouble with cmake -DBUILD_SHARED_LIBS=ON .. step and I get:

cmake -DBUILD_SHARED_LIBS=ON ..
CMake Error at CMakeLists.txt:11 (include):
  include could not find load file:

    Conan

CMake Error at CMakeLists.txt:12 (include):
  include could not find load file:

    BuildType

CMake Error at CMakeLists.txt:13 (include):
  include could not find load file:

    clang-format

CMake Error at CMakeLists.txt:38 (setup_conan):
  Unknown CMake command "setup_conan".

-- Configuring incomplete, errors occurred!
See also "/home/akhailta/work/EDA/Accellera/PySysC-SC/build/CMakeFiles/CMakeOutput.log".

Is there a simple test I can do to see if the PySysC build is successful?

@wlav
Copy link

wlav commented Mar 18, 2021

No idea. :) I've never used PySysC-SC ... (I'm author of cppyy, which is why its mention in a bug report drew me here.)

My assumption is that these packages need to either come from the system packager or from pip. Anyway, if I follow the link you provide above and read the README.md, it states:

pip install conan

and a quick search on pypi.org shows that there is also a package called clang-format. As for BuildType, I don't know ...

@amal-khailtash
Copy link
Author

I am not sure if this is ready for prime-time use. It seems instructions are lacking or not complete. As part of the setup.py, it already installs conan and I am not sure what it is used for. Thanks for looking into this though.

    install_requires=[
        'cppyy',
        'conan'
        ],

@amal-khailtash
Copy link
Author

Almost done compiling, but I still get this when running the example:

$ python router_example.py 
Traceback (most recent call last):
...
    raise RuntimeError("could not load cppyy_backend library, details:\n%s" %
RuntimeError: could not load cppyy_backend library, details:
  .venv/lib/python3.9/site-packages/cppyy_backend/lib/libcppyy_backend.so: undefined symbol: _ZN11CppyyLegacy10TClassEdit9ShortTypeEPKci
  libcppyy_backend.so: cannot open shared object file: No such file or directory
  libcppyy_backend.cpython-39-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory
  .venv/lib/python3.9/site-packages/cppyy_backend/lib/libcppyy_backend.cpython-39-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory

These are installed:

$ pip list | grep cppyy
cppyy              2.1.0
cppyy-backend      1.14.6
cppyy-cling        6.25.1

Not sure what is missing for the cppyy-backend. The shared libraries seem to be in order:

$ LD_LIBRARY_PATH=.venv/lib/python3.9/site-packages/cppyy_backend/lib ldd .venv/lib/python3.9/site-packages/cppyy_backend/lib/*.so

.venv/lib/python3.9/site-packages/cppyy_backend/lib/libCling.so:
	linux-vdso.so.1 =>  (0x00007ffdb9b65000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f6a8e3b8000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f6a8e1b4000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6a8df98000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f6a8dc90000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f6a8d98e000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f6a8d778000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f6a8d3aa000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f6a9276b000)

.venv/lib/python3.9/site-packages/cppyy_backend/lib/libCoreLegacy.so:
	linux-vdso.so.1 =>  (0x00007ffd7bbe5000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fa941674000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa941458000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fa941150000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fa940e4e000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa940c38000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa94086a000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa941b0b000)

.venv/lib/python3.9/site-packages/cppyy_backend/lib/libcppyy_backend.so:
	linux-vdso.so.1 =>  (0x00007ffe2daf8000)
	libstdc++.so.6 => python/3.9.6/lib/libstdc++.so.6 (0x00007f9db5675000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f9db52e3000)
	libgcc_s.so.1 => python/3.9.6/lib/libgcc_s.so.1 (0x00007f9db565f000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9db50c7000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f9db4cf9000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9db5625000)

.venv/lib/python3.9/site-packages/cppyy_backend/lib/libRIOLegacy.so:
	linux-vdso.so.1 =>  (0x00007ffe48d5d000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fee6fec5000)
	libThreadLegacy.so => .venv/lib/python3.9/site-packages/cppyy_backend/lib/libThreadLegacy.so (0x00007fee7055c000)
	libCoreLegacy.so => .venv/lib/python3.9/site-packages/cppyy_backend/lib/libCoreLegacy.so (0x00007fee6fc32000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fee6fa16000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fee6f70e000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fee6f40c000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fee6f1f6000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fee6ee28000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fee703c4000)

.venv/lib/python3.9/site-packages/cppyy_backend/lib/libThreadLegacy.so:
	linux-vdso.so.1 =>  (0x00007ffd284f7000)
	libCoreLegacy.so => .venv/lib/python3.9/site-packages/cppyy_backend/lib/libCoreLegacy.so (0x00007efd7f9e2000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007efd7f7c6000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007efd7f4be000)
	libm.so.6 => /lib64/libm.so.6 (0x00007efd7f1bc000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007efd7efa6000)
	libc.so.6 => /lib64/libc.so.6 (0x00007efd7ebd8000)
	/lib64/ld-linux-x86-64.so.2 (0x00007efd7fc75000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007efd7e9d4000)

@amal-khailtash
Copy link
Author

Using gcc above 5.0 (8.3.0) and doing the following solved this issue:

$ python -m pip install CPyCppyy cppyy-backend --no-cache-dir --upgrade --force-reinstall

But now to a different problem:

$ python router_example.py 
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): center.conan.io:443
DEBUG:urllib3.connectionpool:https://center.conan.io:443 "GET /v1/ping HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://center.conan.io:443 "GET /v1/conans/SystemC/2.3.3/minres/stable/download_urls HTTP/1.1" 404 74
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): artifactory.minres.com:443
DEBUG:urllib3.connectionpool:https://artifactory.minres.com:443 "GET /artifactory/api/conan/oss/v1/ping HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://artifactory.minres.com:443 "GET /artifactory/api/conan/oss/v1/conans/SystemC/2.3.3/minres/stable/download_urls HTTP/1.1" 404 74
...
...
    raise NotFoundException(msg)
conans.errors.NotFoundException: Unable to find 'SystemC/2.3.3@minres/stable' in remotes

@amal-khailtash
Copy link
Author

amal-khailtash commented Oct 15, 2021

I think this is related to the conanfile.txt or artifactory. I am not sure if the artifacts exist in the artifactory.

Can I not point to a previously installed systemc location?

@eyck
Copy link
Contributor

eyck commented Nov 28, 2021

Actually the master branch of https://git.minres.com/SystemC/PySysC-SC/ is out-date with the retirement of the free bintray service. You should try https://git.minres.com/SystemC/PySysC-SC/src/branch/develop. The develop branch with the respective changes has been merged to master.

This also honors now the SYSTEMC_HOME setting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants