Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH add get_worker_rank with unique rank #285

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

tomMoral
Copy link
Collaborator

@tomMoral tomMoral commented Feb 5, 2021

Add loky.get_worker_rank function that returns a unique number between 0 and executor.max_workers - 1 in the workers.
This functionality could be used to better assign resources such as GPU in parallel process.

TODO:

  • implement the rank
  • Add some test that check that it works fine with timeout and resize.
  • Add some doc
  • Add an entry in what's new.

@tomMoral
Copy link
Collaborator Author

tomMoral commented Apr 9, 2024

This PR is another implementation of the feature proposed in #261 .
The main difference is that the map linking pid and worker id is send back with each task, to notify the worker in case a resize occurred and his rank should now be smaller than the initial one.

@tomMoral tomMoral force-pushed the ENH_worker_id branch 3 times, most recently from c8105f7 to 3b68646 Compare April 9, 2024 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant