Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TaskMatcher rewrite [very wip] #7196

Draft
wants to merge 3 commits into
base: priority
Choose a base branch
from
Draft

TaskMatcher rewrite [very wip] #7196

wants to merge 3 commits into from

Conversation

dnr
Copy link
Member

@dnr dnr commented Jan 31, 2025 •
edited
Loading

What changed?

  • Rewrite TaskMatcher to use explicit queues (to be extended into priority queues).
  • taskReader pushes all tasks into TaskMatcher, and loads more when the number outstanding is too low.

This passes all functional tests, but the matching unit tests need work.

Why?

  • Ability to prioritize uniformly across all pending tasks including queries+nexus.
  • Simpler architecture, no new component between taskReader and TaskMatcher.
  • More separated concerns, e.g. "forwarding" is (mostly) in one place instead of spread around, "rate limiting" is in one place, etc.
  • Easier to understand code, no more nested selects (this is very subjective of course).
  • Maybe better performance (after optimizations).

How did you test it?

existing functional tests, need to update unit tests

Potential risks

lots of new code, may be new bugs

@dnr dnr force-pushed the pri3 branch 3 times, most recently from cdebec0 to 8236e0f Compare February 4, 2025 05:04
ShahabT and others added 3 commits February 3, 2025 23:25
@ShahabT @dnr
Fix flaky TestTransitionFromActivity (temporalio#7132)
0c8fbc3 
## What changed?
<!-- Describe what has changed in this PR -->
See title.

## Why?
<!-- Tell your future self why have you made these changes -->
Flaky tests no good!

## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->

## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->

## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->

## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
@hai719 @dnr
fix flaky xdc test TestCronWorkflowCompleteAndFailover (temporalio#7139)
21503b7 
## What changed?
Wait for clusters to be synced instead of using Sleep. Make sure second
run is started before failover.

## Why?
Current implementation depends on the timing which is not reliable.

## How did you test it?
Repeatedly run the test locally and no failure found.

## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->

## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->

## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
@dnr
taskmatcher rewrite wip
a2d0e4c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dnr @ShahabT @hai719