Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sometimes the plugin throws an IndexOutOfBoundsException with multiple locks #662

Open
deiga opened this issue May 30, 2024 · 1 comment
Open

Sometimes the plugin throws an IndexOutOfBoundsException with multiple locks #662

deiga opened this issue May 30, 2024 · 1 comment
Labels
bug Clarification needed Additional clarification needed

Comments

@deiga
Copy link

deiga commented May 30, 2024 •
edited
Loading

Jenkins and plugins versions report

Environment 
Jenkins: 2.426.3
OS: Linux - 5.10.210-201.852.amzn2.x86_64
Java: 17.0.9 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
---
analysis-model-api:12.1.0
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
asm-api:9.7-33.v4d23ef79fcc8
authentication-tokens:1.53.v1c90fd9191a_b_
aws-credentials:231.v08a_59f17d742
aws-java-sdk:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-api-gateway:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-autoscaling:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-cloudformation:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-cloudfront:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-codebuild:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-codedeploy:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-ec2:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-ecr:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-ecs:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-efs:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-elasticbeanstalk:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-elasticloadbalancingv2:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-iam:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-kinesis:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-lambda:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-logs:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-minimal:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-organizations:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-secretsmanager:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-sns:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-sqs:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-ssm:1.12.696-451.v0651a_da_9ca_ec
blueocean:1.27.12
blueocean-bitbucket-pipeline:1.27.12
blueocean-commons:1.27.12
blueocean-config:1.27.12
blueocean-core-js:1.27.12
blueocean-dashboard:1.27.12
blueocean-display-url:2.4.2
blueocean-events:1.27.12
blueocean-git-pipeline:1.27.12
blueocean-github-pipeline:1.27.12
blueocean-i18n:1.27.12
blueocean-jwt:1.27.12
blueocean-personalization:1.27.12
blueocean-pipeline-api-impl:1.27.12
blueocean-pipeline-editor:1.27.12
blueocean-pipeline-scm-api:1.27.12
blueocean-rest:1.27.12
blueocean-rest-impl:1.27.12
blueocean-web:1.27.12
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1152.v6f101e97dd77
build-history-metrics-plugin:112.v476124de7dfc
build-user-vars-plugin:1.9
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.0
cloudbees-bitbucket-branch-source:883.v041fa_695e9c2
cloudbees-disk-usage-simple:203.v3f46a_7462b_1a_
cloudbees-folder:6.858.v898218f3609d
clover:4.14.2.596.vb_4d6475e990b_
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
conditional-buildstep:1.4.3
config-file-provider:968.ve1ca_eb_913f8c
configuration-as-code:1775.v810dc950b_514
copyartifact:722.v0662a_9b_e22a_c
credentials:1337.v60b_d7b_c7b_c9f
credentials-binding:657.v2b_19db_7d6e6d
dark-theme:439.vdef09f81f85e
dashboard-view:2.508.va_74654f026d1
data-tables-api:2.0.3-1
datadog:6.0.3
dependency-check-jenkins-plugin:5.5.0
display-url-api:2.200.vb_9327d658781
docker-commons:439.va_3cb_0a_6a_fb_29
docker-workflow:572.v950f58993843
durable-task:550.v0930093c4b_a_6
echarts-api:5.5.0-1
extended-choice-parameter:381.v360a_25ea_017c
extended-read-permission:53.v6499940139e5
extensible-choice-parameter:1.8.1
external-monitor-job:215.v2e88e894db_f8
favorite:2.208.v91d65b_7792a_c
file-parameters:316.va_83a_1221db_a_7
font-awesome-api:6.5.1-3
forensics-api:2.4.0
git:5.2.1
git-client:4.7.0
git-forensics:2.1.0
git-prebuildmerge-trait:1.0
github:1.38.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1787.v8b_8cd49a_f8f1
github-checks:554.vb_ee03a_000f65
github-oauth:597.ve0c3480fcb_d0
google-oauth-plugin:1.330.vf5e86021cb_ec
gson-api:2.10.1-15.v0d99f670e0a_7
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
htmlpublisher:1.33
http_request:1.18
instance-identity:185.v303dc7c645f9
ionicons-api:70.v2959a_b_74e3cf
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.12
jersey2-api:2.42-147.va_28a_44603b_d5
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
jnr-posix-api:3.1.19-2
jobgenerator:1.22
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery3-api:3.7.1-2
jsch:0.2.16-86.v42e010d9484b_
json-api:20240303-41.v94e11e6de726
json-path-api:2.9.0-58.v62e3e85b_a_655
junit:1265.v65b_14fa_f12f0
kubernetes:4203.v1dd44f5b_1cf9
kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2
kubernetes-credentials:0.11
lockable-resources:1245.vb_05f8a_4e28db_
mailer:470.vc91f60c5d8e2
matrix-auth:3.2.2
matrix-project:822.824.v14451b_c0fd42
metrics:4.2.21-449.v6960d7c54c69
mina-sshd-api-common:2.12.1-101.v85b_e08b_780dd
mina-sshd-api-core:2.12.1-101.v85b_e08b_780dd
mttr:1.1
oauth-credentials:0.646.v02b_66dc03d2e
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
pam-auth:1.10
parameterized-scheduler:262.v00f3d90585cc
parameterized-trigger:787.v665fcf2a_830b_
permissive-script-security:0.7
pipeline-aws:1.45
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github:2.8-159.09e4403bc62f
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-graph-view:215.vc0ea_557e4d6f
pipeline-groovy-lib:704.vc58b_8890a_384
pipeline-input-step:477.v339683a_8d55e
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2198.v41dd8ef6dd56
pipeline-model-definition:2.2198.v41dd8ef6dd56
pipeline-model-extensions:2.2198.v41dd8ef6dd56
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2198.v41dd8ef6dd56
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.2
plain-credentials:179.vc5cb_98f6db_38
plugin-util-api:4.1.0
prism-api:1.29.0-13
project-stats-plugin:23.v47fee1f77b_84
pubsub-light:1.18
run-condition:1.7
saml:4.464.vea_cb_75d7f5e0
scm-api:689.v237b_6d3a_ef7f
script-security:1335.vf07d9ce377a_e
slack:684.v833089650554
snakeyaml-api:2.2-111.vc6598e30cc65
solarized-theme:0.1
sonar:2.17.2
sse-gateway:1.26
ssh-credentials:337.v395d2403ccd4
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
templating-engine:2.5.3
theme-manager:215.vc1ff18d67920
timestamper:1.26
token-macro:400.v35420b_922dcb_
trilead-api:2.142.v748523a_76693
variant:60.v7290fc0eb_b_cd
warnings-ng:11.2.2
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1049.v257a_e6b_30fb_d
workflow-cps:3894.vd0f0248b_a_fc4
workflow-durable-task-step:1331.vc8c2fed35334
workflow-job:1385.vb_58b_86ea_fff1
workflow-multibranch:773.vc4fe1378f1d5
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:657.v03b_e8115821b_
workflow-support:896.v175a_a_9c5b_78f

What Operating System are you using (both controller, and any agents involved in the problem)?

Everything runs in kubernetes. Controller is running the Jenkins docker image and agents run in docker:26.1.1-dind docker image

Reproduction steps

It's hard to determine reproduction steps, since the issue happens randomly (but always on the same pipeline)

We have a Jenkinsfile which determines which locks to request.

extraLocks = []
...
extraLocks.push([resource: "alp-${params.branch}-pr-deployment"])
pipeline {
	options {
        lock(resource: "e2e-${params.branch}-${params.domain}", extra: extraLocks)
    }
}
...

Expected Results

I would expect the plugin to be able to release locks properly at the end of the pipeline and not throw an error.

Actual Results

Start of pipeline:

09:05:27  [Pipeline] lock
09:05:27  Trying to acquire lock on [{e2e-SG1-502-circleci-setup-synapse},{alp-SG1-502-circleci-setup-pr-deployment}]
09:05:27  Resource [e2e-SG1-502-circleci-setup-synapse] did not exist. Created.
09:05:27  [{e2e-SG1-502-circleci-setup-synapse},{alp-SG1-502-circleci-setup-pr-deployment}] is not free, waiting for execution ...
09:09:36  Lock acquired on [{e2e-SG1-502-circleci-setup-synapse},{alp-SG1-502-circleci-setup-pr-deployment}]

End of pipeline:

09:12:51  [Pipeline] End of Pipeline
09:12:51  Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 517bb887-4df9-4e73-a3f3-dd873f5c86b3
09:12:51  java.lang.IndexOutOfBoundsException: Index 2 out of bounds for length 2
09:12:51  	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
09:12:51  	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
09:12:51  	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
09:12:51  	at java.base/java.util.Objects.checkIndex(Objects.java:361)
09:12:51  	at java.base/java.util.ArrayList.get(ArrayList.java:427)
09:12:51  	at org.jenkins.plugins.lockableresources.LockableResourcesManager.getNextQueuedContext(LockableResourcesManager.java:778)
09:12:51  	at org.jenkins.plugins.lockableresources.LockableResourcesManager.proceedNextContext(LockableResourcesManager.java:684)
09:12:51  	at org.jenkins.plugins.lockableresources.LockableResourcesManager.unlockNames(LockableResourcesManager.java:662)
09:12:51  	at org.jenkins.plugins.lockableresources.LockStepExecution$Callback.finished(LockStepExecution.java:224)
09:12:51  	at org.jenkinsci.plugins.workflow.steps.BodyExecutionCallback$TailCall.onSuccess(BodyExecutionCallback.java:118)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$SuccessAdapter.receive(CpsBodyExecution.java:375)
09:12:51  	at com.cloudbees.groovy.cps.Outcome.resumeFrom(Outcome.java:70)
09:12:51  	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:144)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:17)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:49)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:180)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.lambda$wrap$4(CpsVmExecutorService.java:136)
09:12:51  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
09:12:51  	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
09:12:51  	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
09:12:51  	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
09:12:51  	at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
09:12:51  	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
09:12:51  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
09:12:51  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
09:12:51  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.call(CpsVmExecutorService.java:53)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.call(CpsVmExecutorService.java:50)
09:12:51  	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
09:12:51  	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
09:12:51  	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.lambda$categoryThreadFactory$0(CpsVmExecutorService.java:50)
09:12:51  	at java.base/java.lang.Thread.run(Thread.java:840)

Anything else?

No response

Are you interested in contributing a fix?

Sure, if I can get a pointer where to start.
@deiga deiga added the bug label May 30, 2024
@mPokornyETM
Copy link
Contributor

May you pls install newer version.

I provide few fixes in the queue things in the release https://github.com/jenkinsci/lockable-resources-plugin/tree/1246.v28b_e4cc6fa_16

Hopefully it is fixed just now. When not, this "old" stack is helpfull, because the source code looks a "little" bit else now

@mPokornyETM mPokornyETM added the Clarification needed Additional clarification needed label Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Clarification needed Additional clarification needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@deiga @mPokornyETM