-
Notifications
You must be signed in to change notification settings - Fork 273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize gradle-check CI Pipeline to Handle Rapid PR Updates #5008
Comments
@peterzhuamazon @prudhvigodithi @dblock @getsaurabh02 @gaiksaya Appreciate your feedback and comments. |
Thanks a lot @rishabh6788 it totally makes sense to cancel such workflows. It seems cancelling the Jenkins job is the most needed and at the same time, most difficult part of that. However, if we take a step back and tackle splitting Gradle check into isolated tasks / phases, I suspect some of those could be moved from Jenkins to "native" Github action (so we would get the cancellation etc. out of the box). [1] #5010 |
Yeah @rishabh6788 it totally worth invest in this, so we can reduce duplicate runs and save resources. The github actions concurrency settings work out of the box, and I am ok with the custom solutions in jenkinsfile. Assuming no other things needs to be touched, this should be the quickest way to address this issue for now. Thanks. |
100% agree on splitting gradle-check into multiple CI workflows, we can start by just splitting integ-tests and bwc-tests and build upon that. We will be adding this to our sprint and try to prioritize it. Meanwhile the stale/out-dated runs remain a significant problem and cause unnecessary resource consumption, till we have a clear way forward on gradle-check split I will try to solve this problem and once we have a clear vision on split, this can be easily cleaned up. |
Hi @rishabh6788 |
Description:
We're experiencing inefficiencies in our gradle-check CI pipeline due to multiple workflow triggers on rapid PR updates. This results in redundant GitHub Actions runs and Jenkins job executions that cause unneccessary resource consumption and queue exhaustion.
Current Behavior:
Problem:
Desired Behavior:
Proposed Solutions:
At this point we are no longer interested in the outcome of previous instance of the running workflow. Github provides out-of-the-box solution to cancel all the previous running instances the same workflow.
In the example below, the workflow has a concurrency check, where for a particular PR if there is already a github actions workflow running, and new one is submitted, it will auto-cancel the former one.
I am proposing an extra stage in the
gradle-check
jenkinsfile which will run through the builds for the job, check if it is for the same PR, may be based on build description, and abort all the previously running builds.This can be a generic groovy script which can take job name as an argument to be easily extended to other jobs if needed. A common script to handle many other operational tasks or generic checks which could be useful to multiple jobs.
Action Items:
Additional Considerations:
We welcome input from the team on these proposed solutions and any additional ideas to streamline our CI/CD process.
The text was updated successfully, but these errors were encountered: