using Abort phase and cleaning up best-effort #4600
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Daniel Rammer <[email protected]>
dosubot
bot
added
the
size:M
Signed-off-by: Daniel Rammer <[email protected]>
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## bug/abort-map-task-subtasks #4600 +/- ##
===============================================================
+ Coverage 58.95% 59.03% +0.08%
===============================================================
Files 547 484 -63
Lines 42293 41599 -694
===============================================================
- Hits 24934 24560 -374
- Misses 15062 15083 +21
+ Partials 2297 1956 -341
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
pvditt
approved these changes
Dec 14, 2023
There was a problem hiding this comment.
Thanks for putting this up. Introducing a new phase cleans up a lot. Didn't think of not going down the Abort handler path during failure, but since we introduced aborting while running, I agree that we shouldn't attempt aborting again.
I'll need to make some changes to unit tests, but can do that on the other branch.
hamersaw
added a commit
that referenced
this pull request
Dec 15, 2023
* add cleanup on failure for array task permanent failures Signed-off-by: Paul Dittamo <[email protected]> * persist subtask aborts to admin Signed-off-by: Paul Dittamo <[email protected]> * add isAbortedSubtask field to externalResources Signed-off-by: Paul Dittamo <[email protected]> * lint Signed-off-by: Paul Dittamo <[email protected]> * add tests for terminal updates Signed-off-by: Paul Dittamo <[email protected]> * add unit tests for terminatesubtasks Signed-off-by: Paul Dittamo <[email protected]> * lint + update unit test Signed-off-by: Paul Dittamo <[email protected]> * delete comment Signed-off-by: Paul Dittamo <[email protected]> * clean up unit test Signed-off-by: Paul Dittamo <[email protected]> * delete comments Signed-off-by: Paul Dittamo <[email protected]> * set correct error message Signed-off-by: Paul Dittamo <[email protected]> * add new array plugin abort phase Signed-off-by: Paul Dittamo <[email protected]> * update awsbatch array unit test Signed-off-by: Paul Dittamo <[email protected]> * using Abort phase and cleaning up best-effort (#4600) * using Abort phase and cleaning up best-effort Signed-off-by: Daniel Rammer <[email protected]> * removed phase updates on Finalize Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]> * update unit test Signed-off-by: Paul Dittamo <[email protected]> * handling PhaseAbortSubTasks in awsbatch plugin Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Paul Dittamo <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Paul Dittamo <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updating to:
Abortphase directly inExternalResourceInforather than a bool flagAbortSubTaskArrayNodePhaseand otherwise this is attempted duringAbortAbortduring finalize. This function is used to handle finalizer cleanup and resource deletion (if requested), so tasks should not be phase transitioned.