Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dyno: resolve foo() to foo(standalone) if serial version of the iterator is not available. #26023

Merged
merged 12 commits into from
Oct 4, 2024

Conversation

DanilaFe
Copy link
Contributor

@DanilaFe DanilaFe commented Sep 30, 2024

Closes https://github.com/Cray/chapel-private/issues/6720 by rolling in a minor fix (see 8320546)

This PR intends to address a longstanding concern with the leader/follower API design in which to use a parallel iterator, you need to define a serial iterator as well. Thus, you can't iterate over foo() if only foo(standalone) or foo(leader)/foo(follower) are defined. This PR implements this, detecting calls to iterators that were discarded only because of the tag/followThis arguments, and re-trying them with the tags in priority order.

The key observation is that the formal-actual map, when it's checking alignment, is actually very well positioned to determine if a call is valid with the exception of missing a tag argument: it iterates over all formals of a potential candidate, and checks if they have a corresponding actual. Thus, if the following conditions are met, we have candidate that was rejected by may constitute a parallel version of the current call:

  • The candidate is an iterator
  • The only formals that are missing are tag and followThis

If this situation is detected, I store the candidate in a separate list of "rejected iterator candidates". This is intended for future optimization opportunities; see the "Optimization Opportunities" section below. The presence of rejected candidates is propagated to the CallResolutionResult. Other code (currently, the Call* handling in the resolver) can check the CallResolutionResult, and attempt to re-run resolution with iterKind.standalone if it failed before. I added logic that does this, storing the repeated resolution attempts as associated actions.

The fact that all call handling uniformily approaches re-resolving with tag means that parallel iterators are invoked in other contexts than for loops; returning a foo() from a function, or storing it in a variable, is still an acceptable way to invoke a standalone-only version of foo.

Optimization Opportunities

The call resolution process has a lot of steps, including handling last resort candidates, forwarding, and POI scopes. In my opinion, the re-written (foo() -> foo(standalone)) calls should still obey the same rules as all other calls for overloading and candidate selection (meaning they should respect last-resort candidates, "distance", forwarding order etc.). My original intention was to simply treat "fallback" iterator functions as another kind of last resort candidate, and thus have resolveCall succeed for foo() by rewriting the call to foo(standalone) where appropriate. However, this would effectively require a duplication of all handling for last-resort candidates (we'd need "last resort groups" and "last-resort groups of fallback iterators", for example), and generally seems to increase complexity dramatically.

As a result, I went with the "set a flag on CallResolutionResult" approach. Calling code can re-invoke resolveCall with the necessary tag arguments, and call resolution would take care (as it normally does) of precedence, forwarding, receivers, etc. The disadvantage here is that there is likely a lot of overlap when collecting candidates (into a vector of IDs), and that this work is likely performed multiple times in such a case. The rejectedIteratorsMissingTag field is intended as a stub that can be grown if we choose to re-attempt the approach from the previous paragraph, and handle the fallback iterator functions in a single pass.

Reviewed by @dlongnecke-cray -- thanks!

Testing

DanilaFe and others added 12 commits September 30, 2024 16:25
This will help knowing when to re-resolve calls if they fail

Signed-off-by: Danila Fedorin <[email protected]>
@riftEmber
Copy link
Member

Noting this will now also resolve https://github.com/Cray/chapel-private/issues/6720

Copy link
Contributor

@dlongnecke-cray dlongnecke-cray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DanilaFe DanilaFe merged commit 30f2ebc into chapel-lang:main Oct 4, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants