Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi: add timeout to reply.get #221

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

JCourt1
Copy link

@JCourt1 JCourt1 commented Sep 18, 2023

It is possible for this call to hang, like the other calls in this function. Passing through the timeout removes that possibility.

This was first noted on #43

It is possible for this call to hang, like the other calls in this function. Passing through the timeout removes that possibility
Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks a bit like each action is getting 2 spawns (one outer,one inner)

The invocation nesting Looks incorrect

@RonnyPfannschmidt
Copy link
Member

@JCourt1 do you have a example of the hang? the code in question should pass in all cases for threading, is there a different execmodel involved

(the killfunc has a own timeout that should apply)

@JCourt1
Copy link
Author

JCourt1 commented Sep 19, 2023

should pass in all cases for threading

Yes I agree with you in principle, I was confused by this. I can say though that empirically I am hitting the issue... Unfortunately I'm not able to reproduce this in a minimal example outside of the codebase I am encountering it in. It's through pytest-xdist, and I think that does just use "thread" as the execmodel. I just hit it again actually:

  File ".../venv/lib/python3.11/site-packages/xdist/dsession.py", line 87, in pytest_sessionfinish
    nm.teardown_nodes()
  File ".../venv/lib/python3.11/site-packages/xdist/workermanage.py", line 81, in teardown_nodes
    self.group.terminate(self.EXIT_TIMEOUT)
  File ".../venv/lib/python3.11/site-packages/execnet/multi.py", line 214, in terminate
    safe_terminate(
  File ".../venv/lib/python3.11/site-packages/execnet/multi.py", line 310, in safe_terminate
    reply.get()
  File ".../venv/lib/python3.11/site-packages/execnet/gateway_base.py", line 282, in get
    self.waitfinish(timeout)
  File ".../venv/lib/python3.11/site-packages/execnet/gateway_base.py", line 289, in waitfinish
    if not self._result_ready.wait(timeout):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.pyenv/versions/3.11.5/lib/python3.11/threading.py", line 622, in wait
    signaled = self._cond.wait(timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.pyenv/versions/3.11.5/lib/python3.11/threading.py", line 320, in wait
    waiter.acquire()
KeyboardInterrupt

re. this:

This looks a bit like each action is getting 2 spawns (one outer,one inner)

The invocation nesting Looks incorrect

I get the impression that it is that way just to allow the termkills to be run in parallel. But do we even need the for loop over replyList? Could this be deleted:

    for reply in replylist:
        reply.get()

@RonnyPfannschmidt
Copy link
Member

I believe the intent was to complete all Tasks from the pool

Its unclear what's is needed for the other backends

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants