Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NPE/ConcurrentModificationException in JibLayerFilterExtension when building in paralllel large multi-module project #170

Open
DiverOfDark opened this issue Apr 5, 2024 · 4 comments · May be fixed by #178
Open

NPE/ConcurrentModificationException in JibLayerFilterExtension when building in paralllel large multi-module project #170

DiverOfDark opened this issue Apr 5, 2024 · 4 comments · May be fixed by #178
Labels
priority: p3 type: feature New feature or request

Comments

@DiverOfDark
Copy link

Using jib-maven-plugin 3.4.1 with jib-layer-filter-extension-maven 0.3.0 on large multi-module maven project (350 projects, ~50 of them have jib plugin to do jib:build as part of mvn deploy lifecycle). They all share same base image as same jib plugin configuration coming from root pom file.

Whenever I try to build projects in parallel (mvn deploy -T8) - I get these exceptions:

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.1:build (docker-push) on project ***: error running extension 'com.google.cloud.tools.jib.maven.extension.layerfilter.JibLayerFilterExtension': extension crashed: null
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
Caused by: org.apache.maven.plugin.MojoExecutionException: error running extension 'com.google.cloud.tools.jib.maven.extension.layerfilter.JibLayerFilterExtension': extension crashed: null
    at com.google.cloud.tools.jib.maven.BuildImageMojo.execute (BuildImageMojo.java:162)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
Caused by: com.google.cloud.tools.jib.plugins.extension.JibPluginExtensionException: extension crashed: null
    at com.google.cloud.tools.jib.maven.MavenProjectProperties.runPluginExtension (MavenProjectProperties.java:683)
    at com.google.cloud.tools.jib.maven.MavenProjectProperties.runPluginExtensions (MavenProjectProperties.java:621)
    at com.google.cloud.tools.jib.plugins.common.PluginConfigurationProcessor.createJibBuildRunnerForRegistryImage (PluginConfigurationProcessor.java:312)
    at com.google.cloud.tools.jib.maven.BuildImageMojo.execute (BuildImageMojo.java:110)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
Caused by: java.util.ConcurrentModificationException
    at java.util.LinkedHashMap$LinkedHashIterator.nextNode (LinkedHashMap.java:756)
    at java.util.LinkedHashMap$LinkedEntryIterator.next (LinkedHashMap.java:788)
    at java.util.LinkedHashMap$LinkedEntryIterator.next (LinkedHashMap.java:786)
    at com.google.cloud.tools.jib.maven.extension.layerfilter.JibLayerFilterExtension.determineFinalLayerName (JibLayerFilterExtension.java:277)
    at com.google.cloud.tools.jib.maven.extension.layerfilter.JibLayerFilterExtension.extendContainerBuildPlan (JibLayerFilterExtension.java:95)
    at com.google.cloud.tools.jib.maven.MavenProjectProperties.runPluginExtension (MavenProjectProperties.java:675)
    at com.google.cloud.tools.jib.maven.MavenProjectProperties.runPluginExtensions (MavenProjectProperties.java:621)
    at com.google.cloud.tools.jib.plugins.common.PluginConfigurationProcessor.createJibBuildRunnerForRegistryImage (PluginConfigurationProcessor.java:312)
    at com.google.cloud.tools.jib.maven.BuildImageMojo.execute (BuildImageMojo.java:110)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:840)
@suztomo suztomo added type: bug Something isn't working priority: p2 labels Apr 11, 2024
@DiverOfDark
Copy link
Author

Could someone elaborate whether I need to add any further details for this issue to be fixed or to have label for accepting contributions?

@DiverOfDark
Copy link
Author

possible fix could be changing field variables to local in JibLayerFilterExtension:

  Map<PathMatcher, String> pathMatchers = new LinkedHashMap<>();
  Map<String, FileEntriesLayer.Builder> newToLayers = new LinkedHashMap<>();

Issue is if it defined in root pom as part of pluginmanagement - apparently there is only one instance of JibLayerFilterExtension across all modules in such case.

nitschSB added a commit to nitschSB/jib-extensions that referenced this issue May 23, 2024
@nitschSB
remove shared variables
63839c9 
GoogleContainerTools#170
@nitschSB
Copy link

nitschSB commented May 23, 2024 •
edited
Loading

Hi,
we struggle with the same issue.
I've fixed the maven plugin:
master...nitschSB:jib-extensions:issue#170

I don't create a PR. ./CONTRIBUTING.md says

Open a PR only if the issue is labeled with "Accepting Contributions"

(Created a PR anyway)

@GrayYoga
Copy link

Same bug reproduced on jib-maven-plugin:3.4.3 while biuld multimodule project.
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.3:build (default) on project ***: error running extension 'com.google.cloud.tools.jib.maven.extension.layerfilter.JibLayerFilterExtension': extension crashed: null: ConcurrentModificationException -> [Help 1] [ERROR]

@nitschSB nitschSB linked a pull request Jul 17, 2024 that will close this issue
Open
2 tasks
@mpeddada1 mpeddada1 added type: feature New feature or request priority: p3 and removed type: bug Something isn't working priority: p2 labels Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
priority: p3 type: feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix #170: remove shared variables nitschSB/jib-extensions
5 participants
@suztomo @DiverOfDark @GrayYoga @nitschSB @mpeddada1