Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RecursionError: maximum recursion depth exceeded when pickling #118

Open
EwoutH opened this issue Sep 2, 2024 · 3 comments
Open

RecursionError: maximum recursion depth exceeded when pickling #118

EwoutH opened this issue Sep 2, 2024 · 3 comments
Labels
bug Something isn't working dependencies Pull requests that update a dependency file

Comments

@EwoutH
Copy link
Contributor

EwoutH commented Sep 2, 2024

I would like to save the World object to analyze later, but when saving with either:

model1.uw.save("model1_uw.pickle")

or

import sys
sys.setrecursionlimit(1000000)  # Example: Set the limit to 1500, adjust as needed
with open("model_instance.pickle", "wb") as f:
    pickle.dump(model1.uw, f, protocol=pickle.HIGHEST_PROTOCOL)

It throws a RecursionError: maximum recursion depth exceeded message.

Reproduce

I don't have a minimal reproducible example yet, buy it can be observed by running model.py in this repo and adding model1.uw.save("model1_uw.pickle") at the bottom. The error can also be observed in this CI run.

@toruseo toruseo added the bug Something isn't working label Sep 11, 2024
@toruseo
Copy link
Owner

toruseo commented Sep 11, 2024

I believe this is a limitation of dill package for pickling. Is it possible to identify which object is causing this error? It may be possible to remove that object if it was too redundant.

@EwoutH
Copy link
Contributor Author

EwoutH commented Sep 11, 2024

It might be already solved upstream, possibly by uqfoundation/dill#667.

For my own research I found a workaround for now, and it's also not preferable in my case to have multiple 100+ MB model saves.

@toruseo
Copy link
Owner

toruseo commented Oct 3, 2024

New version of dill has been released. Can you check if this error persists?

@toruseo toruseo added the dependencies Pull requests that update a dependency file label Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working dependencies Pull requests that update a dependency file
Projects
None yet
Development

No branches or pull requests

2 participants