Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]: Compatibility Issue with Julia Version When Using PySR #600

Open
ewijaya opened this issue Apr 18, 2024 · 6 comments
Open

[BUG]: Compatibility Issue with Julia Version When Using PySR #600

ewijaya opened this issue Apr 18, 2024 · 6 comments
Assignees
@MilesCranmer
Labels
bug Something isn't working

Comments

@ewijaya
Copy link

ewijaya commented Apr 18, 2024 •
edited
Loading

What happened?

Hi,

I am encountering an issue when attempting to use PySR with a specific version of Julia. Despite setting the JULIA_EXE environment variable to point to my installed Julia version (1.10.0), the PySR setup seems to attempt reinstalling Julia, which results in a version compatibility issue and subsequently fails with an AssertionError.

Environment Details:

  • OS: Ubuntu Linux
  • Python Version: 3.11
  • PySR Version: v0.18.2
  • Julia Version Used: 1.10.0

Steps to Reproduce:

  1. Set the JULIA_EXE environment variable: 
    import os
os.environ["JULIA_EXE"] = "/home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/bin/julia"
  2. Import PySR: 
    from pysr import PySRRegressor

Expected Behavior:
PySR uses the specified version of Julia without attempting to reinstall or validate against other versions, especially since the specified version is already compliant with PySR's requirements.

Actual Behavior:
An error sequence initiates with a message about locating and installing Julia 1.10.0, followed by a compatibility warning and an AssertionError. Here's the relevant part of the traceback:

[juliapkg] Locating Julia ~1.6.7, ~1.7, ~1.8, ~1.9, =1.10.0
[juliapkg] WARNING: You have Julia 1.10.2 installed but ~1.6.7, ~1.7, ~1.8, ~1.9, =1.10.0 is required.
...
AssertionError

Could you please help in resolving this version compatibility issue? It seems there might be a need for clearer handling or documentation regarding the use of specific Julia versions with PySR.

Thank you for your assistance and for developing this useful tool!

Sincerely,
Ed

Version

v0.18.2

Operating System

Linux

Package Manager

None

Interface

IPython Terminal

Relevant log output

In [1]: import os
   ...: # Replace the path below with the exact path to the required Julia version executable
   ...: os.environ["JULIA_EXE"] = "/home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/bin/julia"
   ...:
   ...: # Now, import PySR
   ...: from pysr import PySRRegressor
   ...:
[juliapkg] Locating Julia ~1.6.7, ~1.7, ~1.8, ~1.9, =1.10.0
[juliapkg] Installing Julia 1.10.0 using JuliaUp
[juliapkg] WARNING: You have Julia 1.10.2 installed but ~1.6.7, ~1.7, ~1.8, ~1.9, =1.10.0 is required.
[juliapkg]   It is recommended that you upgrade Julia or install JuliaUp.
[juliapkg] Querying Julia versions from https://julialang-s3.julialang.org/bin/versions.json
[juliapkg] WARNING: About to install Julia 1.10.0 to /home/ubuntu/storage1/anaconda3/julia_env/pyjuliapkg/install.
[juliapkg]   If you use juliapkg in more than one environment, you are likely to have Julia
[juliapkg]   installed in multiple locations. It is recommended to install JuliaUp
[juliapkg]   (https://github.com/JuliaLang/juliaup) or Julia (https://julialang.org/downloads)
[juliapkg]   yourself.
[juliapkg] Downloading Julia from https://julialang-s3.julialang.org/bin/linux/x64/1.10/julia-1.10.0-linux-x86_64.tar.gz
             download complete
[juliapkg] Verifying download
[juliapkg] Installing Julia 1.10.0 to /home/ubuntu/storage1/anaconda3/julia_env/pyjuliapkg/install
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[1], line 6
      3 os.environ["JULIA_EXE"] = "/home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/bin/julia"
      5 # Now, import PySR
----> 6 from pysr import PySRRegressor

File ~/storage1/anaconda3/lib/python3.11/site-packages/pysr/__init__.py:4
      1 # This must be imported as early as possible to prevent
      2 # library linking issues caused by numpy/pytorch/etc. importing
      3 # old libraries:
----> 4 from .julia_import import jl, SymbolicRegression  # isort:skip
      6 from . import sklearn_monkeypatch
      7 from .deprecated import best, best_callable, best_row, best_tex, install, pysr

File ~/storage1/anaconda3/lib/python3.11/site-packages/pysr/julia_import.py:38
     30     for k, default in (
     31         ("PYTHON_JULIACALL_HANDLE_SIGNALS", "yes"),
     32         ("PYTHON_JULIACALL_THREADS", "auto"),
     33         ("PYTHON_JULIACALL_OPTLEVEL", "3"),
     34     ):
     35         os.environ[k] = os.environ.get(k, default)
---> 38 from juliacall import Main as jl  # type: ignore
     40 jl_version = (jl.VERSION.major, jl.VERSION.minor, jl.VERSION.patch)
     42 # Next, automatically load the juliacall extension if we're in a Jupyter notebook

File ~/storage1/anaconda3/lib/python3.11/site-packages/juliacall/__init__.py:251
    234         if Base.Threads.nthreads() > 1:
    235             # a warning to help multithreaded users
    236             # TODO: should we set PYTHON_JULIACALL_HANDLE_SIGNALS=yes whenever PYTHON_JULIACALL_THREADS != 1?
    237             warnings.warn(
    238                 "Julia was started with multiple threads "
    239                 "but multithreading support is experimental in JuliaCall. "
   (...)
    248                 "PYTHON_JULIACALL_HANDLE_SIGNALS=no."
    249             )
--> 251 init()
    253 def load_ipython_extension(ip):
    254     import juliacall.ipython

File ~/storage1/anaconda3/lib/python3.11/site-packages/juliacall/__init__.py:158, in init()
    155 import juliapkg
    157 # Find the Julia executable and project
--> 158 CONFIG['exepath'] = exepath = juliapkg.executable()
    159 CONFIG['project'] = project = juliapkg.project()
    161 # Find the Julia library

File ~/storage1/anaconda3/lib/python3.11/site-packages/juliapkg/deps.py:303, in executable()
    302 def executable():
--> 303     resolve()
    304     return STATE['executable']

File ~/storage1/anaconda3/lib/python3.11/site-packages/juliapkg/deps.py:241, in resolve(force, dry_run)
    239 # find a compatible julia executable
    240 log(f'Locating Julia{"" if compat is None else " "+str(compat)}')
--> 241 exe, ver = find_julia(compat=compat, prefix=STATE['install'], install=True, upgrade=True)
    242 log(f'Using Julia {ver} at {exe}')
    243 # set up the project

File ~/storage1/anaconda3/lib/python3.11/site-packages/juliapkg/find_julia.py:91, in find_julia(compat, prefix, install, upgrade)
     89 pr_ver = julia_version(pr_exe)
     90 assert pr_ver is not None
---> 91 assert compat is None or pr_ver in compat
     92 assert bestcompat is None or pr_ver in bestcompat
     93 return (pr_exe, pr_ver)

AssertionError:

Extra Info

No response
@ewijaya ewijaya added the bug Something isn't working label Apr 18, 2024
@MilesCranmer
Copy link
Owner

I think the right variable is PYTHON_JULIACALL_BINDIR and should pass the directory of the julia binary: https://juliapy.github.io/PythonCall.jl/stable/juliacall/#julia-config

Maybe try that instead and see if it works?

@MilesCranmer MilesCranmer mentioned this issue Apr 18, 2024
Open
@ewijaya
Copy link
Author

ewijaya commented Apr 19, 2024

I think the right variable is PYTHON_JULIACALL_BINDIR and should pass the directory of the julia binary: https://juliapy.github.io/PythonCall.jl/stable/juliacall/#julia-config

Maybe try that instead and see if it works?

Hi Miles,

Thanks for you reply. I tried, but get another error:

In [1]: import os

In [2]: os.environ["PYTHON_JULIACALL_BINDIR"] = "/home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/bin/"

In [3]: from pysr import PySRRegressor
[juliapkg] Locating Julia ~1.6.7, ~1.7, ~1.8, ~1.9, =1.10.0
[juliapkg] Querying Julia versions from https://julialang-s3.julialang.org/bin/versions.json
[juliapkg] Using Julia 1.10.0 at /home/ubuntu/storage1/anaconda3/julia_env/pyjuliapkg/install/bin/julia
[juliapkg] Using Julia project at /home/ubuntu/storage1/anaconda3/julia_env
[juliapkg] Installing packages:
           julia> import Pkg
           julia> Pkg.Registry.update()
           julia> Pkg.add([Pkg.PackageSpec(name="SymbolicRegression", uuid="8254be44-1295-4e6a-a16d-46603ac705cb"), Pkg.PackageSpec(name="Serialization", uuid="9e88b42a-f829-5b0c-bbe9-9e923198166b"), Pkg.PackageSpec(name="PythonCall", uuid="6099a3de-0909-46bc-b1f4-468b9a2dfc0d")])
           julia> Pkg.resolve()
           julia> Pkg.precompile()
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/storage1/anaconda3/julia_env/Project.toml`
  [6099a3de] + PythonCall v0.9.19
  [8254be44] + SymbolicRegression v0.24.2
    Updating `~/storage1/anaconda3/julia_env/Manifest.toml`
  [79e6a3ab] + Adapt v4.0.4
  [4fba245c] + ArrayInterface v7.9.0
  [bbf7d656] + CommonSubexpressions v0.3.0
  [34da2185] + Compat v4.14.0
  [992eb4ea] + CondaPkg v0.2.22
  [187b0558] + ConstructionBase v1.5.5
  [9a962f9c] + DataAPI v1.16.0
  [864edb3b] + DataStructures v0.18.20
  [e2d170a0] + DataValueInterfaces v1.0.0
  [163ba53b] + DiffResults v1.1.0
  [b552c78f] + DiffRules v1.15.1
  [ffbed154] + DocStringExtensions v0.9.3
  [a40a106e] + DynamicExpressions v0.16.0
  [06fc5a27] + DynamicQuantities v0.13.2
  [1a297f60] + FillArrays v1.10.0
  [6a86dc24] + FiniteDiff v2.23.1
  [f6369f11] + ForwardDiff v0.10.36
  [92d709cd] + IrrationalConstants v0.2.2
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.5.0
  [0f8b85d8] + JSON3 v1.14.0
  [d3d80556] + LineSearches v7.2.0
  [2ab3a3ac] + LogExpFunctions v0.3.27
  [30fc2ffe] + LossFunctions v0.11.1
  [e80e1ace] + MLJModelInterface v1.9.6
  [1914dd2f] + MacroTools v0.5.13
  [0b3b1443] + MicroMamba v0.1.14
  [e1d29d7a] + Missings v1.2.0
  [d41bc354] + NLSolversBase v7.8.3
  [77ba4419] + NaNMath v1.0.2
  [429524aa] + Optim v1.9.4
  [bac558e1] + OrderedCollections v1.6.3
  [65ce6f38] + PackageExtensionCompat v1.0.2
  [d96e819e] + Parameters v0.12.3
  [69de0a69] + Parsers v2.8.1
  [fa939f87] + Pidfile v1.3.0
  [85a6dd25] + PositiveFactorizations v0.2.4
  [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.4.3
  [49802e3a] + ProgressBars v1.5.1
  [6099a3de] + PythonCall v0.9.19
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.0
  [30f210dd] + ScientificTypesBase v3.0.0
  [6c6a2e73] + Scratch v1.2.1
  [efcf1570] + Setfield v1.1.1
  [a2af1166] + SortingAlgorithms v1.2.1
  [276daf66] + SpecialFunctions v2.3.1
  [1e83bf80] + StaticArraysCore v1.4.2
  [64bff920] + StatisticalTraits v3.2.0
  [82ae8749] + StatsAPI v1.7.0
  [2913bbd2] + StatsBase v0.34.3
  [856f2bd8] + StructTypes v1.10.0
  [8254be44] + SymbolicRegression v0.24.2
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.11.1
  [410a4b4d] + Tricks v0.1.8
  [3a884ed6] + UnPack v1.0.2
  [e17b2a0c] + UnsafePointers v1.0.0
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [f8abcde7] + micromamba_jll v1.5.8+0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [9fa8497b] + Future
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.10.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays v1.10.0
  [10745b16] + Statistics v1.10.0
  [4607b0f0] + SuiteSparse
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.0.5+1
  [deac9b47] + LibCURL_jll v8.4.0+0
  [e37daf67] + LibGit2_jll v1.6.4+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.2+1
  [14a3606d] + MozillaCACerts_jll v2023.1.10
  [4536629a] + OpenBLAS_jll v0.3.23+2
  [05823500] + OpenLibm_jll v0.8.1+2
  [bea87d4a] + SuiteSparse_jll v7.2.1+1
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850b90] + libblastrampoline_jll v5.8.0+1
  [8e850ede] + nghttp2_jll v1.52.0+1
  [3f19e933] + p7zip_jll v17.4.0+2
  No Changes to `~/storage1/anaconda3/julia_env/Project.toml`
  No Changes to `~/storage1/anaconda3/julia_env/Manifest.toml`

[3793] signal (11.1): Segmentation fault
in expression starting at none:2
initialize_env at /workspace/srcdir/gcc-13.2.0/libgomp/env.c:2062
unknown function (ip: 0x7f1e8ef56b99)
unknown function (ip: 0x7f1e8ef56ca0)
_dl_catch_exception at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f1e8ef5b0ce)
_dl_catch_exception at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f1e8ef5a609)
unknown function (ip: 0x7f1e8eef934b)
_dl_catch_exception at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_dl_catch_error at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f1e8eef9b58)
dlopen at /lib/x86_64-linux-gnu/libdl.so.2 (unknown line)
ijl_load_dynamic_library at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/dlload.c:365
#dlopen#3 at ./libdl.jl:117
dlopen at ./libdl.jl:116 [inlined]
dlopen at ./libdl.jl:116
jfptr_dlopen_51523.1 at /home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
__init__ at /home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/share/julia/stdlib/v1.10/CompilerSupportLibraries_jll/src/CompilerSupportLibraries_jll.jl:62
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_module_run_initializer at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:76
run_module_init at ./loading.jl:1128
register_restored_modules at ./loading.jl:1116
_include_from_serialized at ./loading.jl:1061
_tryrequire_from_serialized at ./loading.jl:1525
unknown function (ip: 0x7f1e88db625f)
_require at ./loading.jl:1984
__require_prelocked at ./loading.jl:1806
jfptr___require_prelocked_80742.1 at /home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_in_world at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/builtins.c:831
#invoke_in_world#3 at ./essentials.jl:921 [inlined]
invoke_in_world at ./essentials.jl:918 [inlined]
_require_prelocked at ./loading.jl:1797
macro expansion at ./loading.jl:1784 [inlined]
macro expansion at ./lock.jl:267 [inlined]
__require at ./loading.jl:1747
jfptr___require_80707.1 at /home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_in_world at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/builtins.c:831
#invoke_in_world#3 at ./essentials.jl:921 [inlined]
invoke_in_world at ./essentials.jl:918 [inlined]
require at ./loading.jl:1740
jfptr_require_80704.1 at /home/ubuntu/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_call at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:126
eval_value at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:223
eval_stmt_value at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:174 [inlined]
eval_body at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:617
eval_body at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:544
jl_interpret_toplevel_thunk at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:775
jl_toplevel_eval_flex at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:934
jl_toplevel_eval_flex at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:877
ijl_toplevel_eval_in at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:985
ijl_eval_string at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/jlapi.c:117
ffi_call_unix64 at /home/ubuntu/storage1/anaconda3/lib/python3.11/lib-dynload/../../libffi.so.8 (unknown line)
ffi_call_int at /home/ubuntu/storage1/anaconda3/lib/python3.11/lib-dynload/../../libffi.so.8 (unknown line)
ffi_call at /home/ubuntu/storage1/anaconda3/lib/python3.11/lib-dynload/../../libffi.so.8 (unknown line)
_call_function_pointer at /usr/local/src/conda/python-3.11.5/Modules/_ctypes/callproc.c:923 [inlined]
_ctypes_callproc at /usr/local/src/conda/python-3.11.5/Modules/_ctypes/callproc.c:1262
PyCFuncPtr_call at /usr/local/src/conda/python-3.11.5/Modules/_ctypes/_ctypes.c:4201
_PyObject_MakeTpCall at /usr/local/src/conda/python-3.11.5/Objects/call.c:214
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:4774
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439
PyEval_EvalCode at /usr/local/src/conda/python-3.11.5/Python/ceval.c:1154
builtin_exec_impl at /usr/local/src/conda/python-3.11.5/Python/bltinmodule.c:1077 [inlined]
builtin_exec at /usr/local/src/conda/python-3.11.5/Python/clinic/bltinmodule.c.h:465
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.5/Objects/methodobject.c:443
do_call_core at /usr/local/src/conda/python-3.11.5/Python/ceval.c:7329 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:5381
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.5/Objects/call.c:393
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_call.h:92 [inlined]
object_vacall at /usr/local/src/conda/python-3.11.5/Objects/call.c:819
PyObject_CallMethodObjArgs at /usr/local/src/conda/python-3.11.5/Objects/call.c:879
import_find_and_load at /usr/local/src/conda/python-3.11.5/Python/import.c:1737 [inlined]
PyImport_ImportModuleLevelObject at /usr/local/src/conda/python-3.11.5/Python/import.c:1836
import_name at /usr/local/src/conda/python-3.11.5/Python/ceval.c:7429 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:3951
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439
PyEval_EvalCode at /usr/local/src/conda/python-3.11.5/Python/ceval.c:1154
builtin_exec_impl at /usr/local/src/conda/python-3.11.5/Python/bltinmodule.c:1077 [inlined]
builtin_exec at /usr/local/src/conda/python-3.11.5/Python/clinic/bltinmodule.c.h:465
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.5/Objects/methodobject.c:443
do_call_core at /usr/local/src/conda/python-3.11.5/Python/ceval.c:7329 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:5381
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.5/Objects/call.c:393
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_call.h:92 [inlined]
object_vacall at /usr/local/src/conda/python-3.11.5/Objects/call.c:819
PyObject_CallMethodObjArgs at /usr/local/src/conda/python-3.11.5/Objects/call.c:879
import_find_and_load at /usr/local/src/conda/python-3.11.5/Python/import.c:1737 [inlined]
PyImport_ImportModuleLevelObject at /usr/local/src/conda/python-3.11.5/Python/import.c:1836
import_name at /usr/local/src/conda/python-3.11.5/Python/ceval.c:7429 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:3951
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439
PyEval_EvalCode at /usr/local/src/conda/python-3.11.5/Python/ceval.c:1154
builtin_exec_impl at /usr/local/src/conda/python-3.11.5/Python/bltinmodule.c:1077 [inlined]
builtin_exec at /usr/local/src/conda/python-3.11.5/Python/clinic/bltinmodule.c.h:465
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.5/Objects/methodobject.c:443
do_call_core at /usr/local/src/conda/python-3.11.5/Python/ceval.c:7329 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:5381
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.5/Objects/call.c:393
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_call.h:92 [inlined]
object_vacall at /usr/local/src/conda/python-3.11.5/Objects/call.c:819
PyObject_CallMethodObjArgs at /usr/local/src/conda/python-3.11.5/Objects/call.c:879
import_find_and_load at /usr/local/src/conda/python-3.11.5/Python/import.c:1737 [inlined]
PyImport_ImportModuleLevelObject at /usr/local/src/conda/python-3.11.5/Python/import.c:1836
import_name at /usr/local/src/conda/python-3.11.5/Python/ceval.c:7429 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:3951
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439
PyEval_EvalCode at /usr/local/src/conda/python-3.11.5/Python/ceval.c:1154
builtin_exec_impl at /usr/local/src/conda/python-3.11.5/Python/bltinmodule.c:1077 [inlined]
builtin_exec at /usr/local/src/conda/python-3.11.5/Python/clinic/bltinmodule.c.h:465
cfunction_vectorcall_FASTCALL_KEYWORDS at /usr/local/src/conda/python-3.11.5/Objects/methodobject.c:443
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_call.h:92 [inlined]
PyObject_Vectorcall at /usr/local/src/conda/python-3.11.5/Objects/call.c:299
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:4774
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
gen_send_ex2 at /usr/local/src/conda/python-3.11.5/Objects/genobject.c:219
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:2591
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
gen_send_ex2 at /usr/local/src/conda/python-3.11.5/Objects/genobject.c:219
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:2591
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
gen_send_ex2 at /usr/local/src/conda/python-3.11.5/Objects/genobject.c:219
gen_send_ex at /usr/local/src/conda/python-3.11.5/Objects/genobject.c:287
method_vectorcall_O at /usr/local/src/conda/python-3.11.5/Objects/descrobject.c:481
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_call.h:92 [inlined]
PyObject_Vectorcall at /usr/local/src/conda/python-3.11.5/Objects/call.c:299
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:4774
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439 [inlined]
_PyFunction_Vectorcall at /usr/local/src/conda/python-3.11.5/Objects/call.c:393 [inlined]
_PyObject_VectorcallTstate at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_call.h:92
method_vectorcall at /usr/local/src/conda/python-3.11.5/Objects/classobject.c:59
_PyVectorcall_Call at /usr/local/src/conda/python-3.11.5/Objects/call.c:257 [inlined]
_PyObject_Call at /usr/local/src/conda/python-3.11.5/Objects/call.c:328 [inlined]
PyObject_Call at /usr/local/src/conda/python-3.11.5/Objects/call.c:355
do_call_core at /usr/local/src/conda/python-3.11.5/Python/ceval.c:7357 [inlined]
_PyEval_EvalFrameDefault at /usr/local/src/conda/python-3.11.5/Python/ceval.c:5381
_PyEval_EvalFrame at /usr/local/src/conda/python-3.11.5/Include/internal/pycore_ceval.h:73 [inlined]
_PyEval_Vector at /usr/local/src/conda/python-3.11.5/Python/ceval.c:6439
PyEval_EvalCode at /usr/local/src/conda/python-3.11.5/Python/ceval.c:1154
run_eval_code_obj at /usr/local/src/conda/python-3.11.5/Python/pythonrun.c:1712
run_mod at /usr/local/src/conda/python-3.11.5/Python/pythonrun.c:1733
pyrun_file at /usr/local/src/conda/python-3.11.5/Python/pythonrun.c:1628
_PyRun_SimpleFileObject at /usr/local/src/conda/python-3.11.5/Python/pythonrun.c:440
_PyRun_AnyFileObject at /usr/local/src/conda/python-3.11.5/Python/pythonrun.c:79
pymain_run_file_obj at /usr/local/src/conda/python-3.11.5/Modules/main.c:360 [inlined]
pymain_run_file at /usr/local/src/conda/python-3.11.5/Modules/main.c:379 [inlined]
pymain_run_python at /usr/local/src/conda/python-3.11.5/Modules/main.c:601 [inlined]
Py_RunMain at /usr/local/src/conda/python-3.11.5/Modules/main.c:680
Py_BytesMain at /usr/local/src/conda/python-3.11.5/Modules/main.c:734
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /home/ubuntu/storage1/anaconda3/bin/python (unknown line)
Allocations: 2906 (Pool: 2897; Big: 9); GC: 0
[1]    3793 segmentation fault (core dumped)  ipython

Sincerely,
Ed

@MilesCranmer
Copy link
Owner

Strange... What happens if you just do

julia --project=~/storage1/anaconda3/julia_env/ -e 'using Pkg; Pkg.instantiate(); Pkg.precompile()'

This should do it directly on the Julia side. If it segfaults there then it might point to a different problem instead.

@MilesCranmer
Copy link
Owner

MilesCranmer commented Apr 19, 2024 •
edited
Loading

Might be good to also raise this on PythonCall.jl issues: https://github.com/JuliaPy/PythonCall.jl since it is segfaulting right at import (so is likely an issue on PythonCall.jl rather than PySR). It will likely get good insights from the maintainer of that package. But happy to try to fix it here simultaneously.

@ewijaya
Copy link
Author

ewijaya commented Apr 19, 2024

Strange... What happens if you just do

julia --project=~/storage1/anaconda3/julia_env/ -e 'using Pkg; Pkg.instantiate(); Pkg.precompile()'

This should do it directly on the Julia side. If it segfaults there then it might point to a different problem instead.

I tried your command. It has no problem:

$ julia --project=~/storage1/anaconda3/julia_env/ -e 'using Pkg; Pkg.instantiate(); Pkg.precompile()'
Precompiling project...
  17 dependencies successfully precompiled in 62 seconds. 61 already precompiled.

@MilesCranmer
Copy link
Owner

So it looks like it compiled them all which is interesting, so at least part of it worked. That julia you are running is the one from your BINDIR right?

Can you run juliacall directly? Like just do:

python -c 'from juliacall import Main as jl; jl.println("Hello World!")'

It would also be interesting to hear if other Python versions work for you or not... I kind of wonder if it's some glibc issue in your environment.

@sefffal sefffal mentioned this issue Nov 7, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MilesCranmer MilesCranmer

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MilesCranmer @ewijaya