Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradle upgrade superset including Jenkins work and more! #3835

Merged
merged 35 commits into from
Mar 3, 2020

Conversation

Cervator
Copy link
Member

@Cervator Cervator commented Feb 23, 2020

I've built this primarily on #3806 with great thanks to @DarkWeird !

It finishes some of the more quirky parts of the overall logistics, does other minor stuff, enhances support for testing things outside our "live" environment in Artifactory, Jenkins, and on GitHub, updates to use Jenkins pipelines properly and the Jenkins Templating Engine (for modules). Also pulled in a couple other PRs and tested other stuff yet.

I believe this is merge-ready now and have new build jobs in the CBC Jenkins queued up to then start building everything fresh and neat in the new Jenkins, including a working Omega zip pulling in modules from new GitHub org mega-jobs automatically created by scanning for repos on GitHub - no more manually jobs in Jenkins and PRs will get tested for modules as well as the engine!

There are still improvements to make before we can use the new system for releases, primarily the approach remains hard coded to use the engine develop branch and snapshot versions for modules (much like before). Some tweaking needs to happen like consider develop a producer of snapshots and master a producer of released artifacts.. Not hard, just more work to do.

Hard to directly test some of this since Jenkins, but see #3806 for some of the central details and try to find any sort of Gradle task that appears broken. It is quite possible there could still be edge cases related to more rarely used stuff or rare combos involving user environments.

DarkWeird and others added 30 commits December 13, 2019 21:24
Fix tests after gradle upgrade
Fix gather libraries for distribution
Fix. move -Xmx options to jvmArgs
…n remapper (FacadeServer cannot building in workspace)
Move workspace features from module to workspace
Unify plugin style
@Cervator Cervator added the Category: Build/CI Requests, Issues and Changes targeting gradle, groovy, Jenkins, etc. label Feb 23, 2020
@Cervator Cervator added this to the v2.3.0 milestone Feb 23, 2020
@GooeyHub
Copy link
Member

Hooray Jenkins reported success with all tests good!


// Magic for replace remote dependency on local project (source)
// For exists modules
project(":modules").subprojects.forEach { proj ->
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block appears to be what can cause a stall with gradlew idea when org.gradle.parallel=true is set in gradle.properties - FYI @DarkWeird

If I comment this block out then the stall doesn't happen (but I'd assume module dependencies won't work). Probably relates to multiple threads working different modules for the ideaModule task yet this block iterates through all modules and for each goes and looks for every other module and substitutes stuff? That seems a classic case for ConcurrentModificationException being thrown when working with a Map just to come up with a possibly similar example :-)

We could still merge and advise to keep parallel processing off, with the intent to later get rid of gradlew idea entirely - but there could be a possibility of similar stalls in other Gradle logic that needs to iterate over modules?

Running Gradle in parallel mode is nice for efficiency but probably not really high priority yet. Could be more of an optimization thing to review later :-)

Copy link

@syntaxi syntaxi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested out and this works almost fine for me.
Removing org.gradle.parallel=true from gradle.properties does fix the issue.

The additional caveat now is that when running gradlew cleanIdea idea (and possibly just idea) you need to have close the project in IntelliJ. Once the command is finished reopening will have it work properly but it doesn't correctly setup on configs if the project is still open.
I don't think this is too much of an issue, as this used to be how the project function before we switched some things over to groovyw.

I think that after this PR we should also take a look at getting the IntelliJ gradle importing working because I suspect this would fix that issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category: Build/CI Requests, Issues and Changes targeting gradle, groovy, Jenkins, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants