Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Addressing multiprocessing DeprecationWarning with Python 3.12 #656

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

GillesOrban
Copy link
Contributor

See issue #655. Defaulting to 'forkserver' and avoiding 'fork'
In addition to fixing the DeprecationWarning, it should also makes operation safer (personally, it solves an issue I had when launching several jobs).

@VChristiaens
Copy link
Contributor

Thanks @GillesOrban. I had started implementing the same solution but encountered the same problems as above when running the automated tests: global variables are inherited by the child processes with the 'fork' start method, but not with the 'spawn' method, and not by default with the 'forkserver' methods. This leads to some undefined variables in the tests.
So I first wanted to check for a workaround for the different global variables that are defined in post-processing routines to not cause a problem with the 'forkserver' or 'spawn' methods before proposing a solution. Searching a bit I found that set_forkserver_preload(modules_names) may work, but would require some more testing.
If this doesn't work, the solution may be to re-define and propagate these variables in each subroutine to avoid the issue. Since these global variables were one of the last changes that @carlos-gg did before he left astronomy, I wonder whether the idea back then was precisely to boost multiprocessing speed with the 'fork' method which we'd be undoing here (albeit motivated by deprecation issues).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants