Unknown failure uploading files to threads

[dclyde-brg]

Jenkins and plugins versions report

Environment

Jenkins: 2.462.2
OS: Linux - 6.1.109-118.189.amzn2023.x86_64
Java: 17.0.12 - Amazon.com Inc. (OpenJDK 64-Bit Server VM)
---
analysis-model-api:12.5.0
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
apache-httpcomponents-client-5-api:5.3.1-117.v4d95117cd34f
asm-api:9.7-33.v4d23ef79fcc8
authentication-tokens:1.119.v50285141b_7e1
authorize-project:1.7.2
aws-credentials:231.v08a_59f17d742
aws-java-sdk-cloudformation:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-codebuild:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-ec2:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-ecr:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-ecs:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-efs:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-elasticbeanstalk:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-iam:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-kinesis:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-logs:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-minimal:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-secretsmanager:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-sns:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-sqs:1.12.767-467.vb_e93f0c614b_6
aws-java-sdk-ssm:1.12.767-467.vb_e93f0c614b_6
blueocean:1.27.16
blueocean-bitbucket-pipeline:1.27.16
blueocean-commons:1.27.16
blueocean-config:1.27.16
blueocean-core-js:1.27.16
blueocean-dashboard:1.27.16
blueocean-display-url:2.4.3
blueocean-events:1.27.16
blueocean-git-pipeline:1.27.16
blueocean-github-pipeline:1.27.16
blueocean-i18n:1.27.16
blueocean-jwt:1.27.16
blueocean-personalization:1.27.16
blueocean-pipeline-api-impl:1.27.16
blueocean-pipeline-editor:1.27.16
blueocean-pipeline-scm-api:1.27.16
blueocean-rest:1.27.16
blueocean-rest-impl:1.27.16
blueocean-web:1.27.16
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_
branch-api:2.1178.v969d9eb_c728e
build-name-setter:2.4.3
build-timeout:1.33
build-user-vars-plugin:176.vb_9c7907fd524
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.1
cloudbees-bitbucket-branch-source:888.v8e6d479a_1730
cloudbees-folder:6.951.v5f91d88d76b_b_
command-launcher:115.vd8b_301cc15d0
commons-compress-api:1.26.1-2
commons-lang3-api:3.17.0-84.vb_b_938040b_078
commons-text-api:1.12.0-129.v99a_50df237f7
conditional-buildstep:1.4.3
configuration-as-code:1850.va_a_8c31d3158b_
copyartifact:749.vfb_dca_a_9b_6549
credentials:1378.v81ef4269d764
credentials-binding:681.vf91669a_32e45
dashboard-view:2.517.v776a_b_811a_b_4e
data-tables-api:2.1.6-1
display-url-api:2.204.vf6fddd8a_8b_e9
durable-task:577.v2a_8a_4b_7c0247
ec2:1688.v8c07e01d657f
echarts-api:5.5.1-1
eddsa-api:0.3.0-4.v84c6f0f4969e
embeddable-build-status:487.va_0ef04c898a_2
envinject:2.919.v009a_a_1067cd0
envinject-api:1.199.v3ce31253ed13
favorite:2.221.v19ca_666b_62f5
file-operations:266.v9d4e1eb_235b_a_
font-awesome-api:6.6.0-2
forensics-api:2.5.0
git:5.4.1
git-client:5.0.0
github:1.40.0
github-api:1.321-468.v6a_9f5f2d5a_7e
github-branch-source:1797.v86fdb_4d57d43
gson-api:2.11.0-41.v019fcf6125dc
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
htmlpublisher:1.36
instance-identity:185.v303dc7c645f9
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-10
jaxb:2.3.9-1
jdk-tool:80.v8a_dee33ed6f0
jenkins-design-language:1.27.16
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
job-dsl:1.89
jobConfigHistory:1264.v054b_c928547e
joda-time-api:2.13.0-85.vb_64d1c2921f1
jquery3-api:3.7.1-2
jsch:0.2.16-86.v42e010d9484b_
json-api:20240303-41.v94e11e6de726
json-path-api:2.9.0-58.v62e3e85b_a_655
junit:1300.v03d9d8a_cf1fb_
mailer:472.vf7c289a_4b_420
markdown-formatter:220.v1a_262cd9f77f
matrix-auth:3.2.2
matrix-project:832.va_66e270d2946
mina-sshd-api-common:2.13.2-125.v200281b_61d59
mina-sshd-api-core:2.13.2-125.v200281b_61d59
node-iterator-api:55.v3b_77d4032326
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
p4:1.16.0
parameter-separator:166.vd0120849b_386
parameterized-scheduler:277.v61a_4b_a_49a_c5c
parameterized-trigger:806.vf6fff3e28c3e
permissive-script-security:0.7
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-groovy-lib:730.ve57b_34648c63
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-definition:2.2214.vb_b_34b_2ea_9b_83
pipeline-model-extensions:2.2214.vb_b_34b_2ea_9b_83
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2214.vb_b_34b_2ea_9b_83
pipeline-stage-view:2.34
pipeline-utility-steps:2.17.0
plain-credentials:183.va_de8f1dd5a_2b_
plugin-util-api:4.1.0
prism-api:1.29.0-17
pubsub-light:1.18
rebuild:332.va_1ee476d8f6d
resource-disposer:0.23
role-strategy:743.v142ea_b_d5f1d3
run-condition:1.7
scm-api:696.v778d637b_a_762
script-security:1362.v67dc1f0e1b_b_3
slack:741.v00f9591c586d
snakeyaml-api:2.3-123.v13484c65210a_
sse-gateway:1.27
ssh-agent:376.v8933585c69d3
ssh-credentials:343.v884f71d78167
ssh-steps:2.0.68.va_d21a_12a_6476
sshd:3.330.vc866a_8389b_58
structs:338.v848422169819
throttle-concurrents:2.14
timestamper:1.27
token-macro:400.v35420b_922dcb_
trilead-api:2.147.vb_73cc728a_32e
variant:60.v7290fc0eb_b_cd
warnings-ng:11.6.0
workflow-aggregator:600.vb_57cdd26fdd7
workflow-api:1336.vee415d95c521
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:3964.v0767b_4b_a_0b_fa_
workflow-durable-task-step:1371.vb_7cec8f3b_95e
workflow-job:1436.vfa_244484591f
workflow-multibranch:795.ve0cb_1f45ca_9a_
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:678.v3ee58b_469476
workflow-support:926.v9f4f9b_b_98c19

What Operating System are you using (both controller, and any agents involved in the problem)?

Windows Controller
Windows Agent

Reproduction steps

def fileName = "test.txt";
writeFile file: fileName, text: "Test File"
    				
def channelName = 'test-builds'
    				
// This posts a new message with the file but lacking any jenkins configured icon/username overrides
slackUploadFile(channel: channelName, filePath: fileName, initialComment: 'Original Message') 

// Normal new message, respects jenkins configured icon/username overrides
def message = slackSend(channel: channelName, message: 'Test Thread')

// Fails to upload file
slackUploadFile(channel: message.threadId, filePath: fileName, initialComment: 'Using ThreadId')

// Says it may have failed to  upload file due to not finding the channel, but it does show up
slackUploadFile(channel: message.channelId, filePath: fileName, initialComment: 'Using Channel Id')

Expected Results

The above code should create a small file and upload it three different ways:

1. As a message in the channel 'test-builds' with the text 'Original Message'
2. As a reply 'Using Thread Id' to message 'Test Thread'
3. As a message in the channel 'test-builds' with the text 'Using Channel Id'

It is expected that there is no error output
It is expected the messages/threads should appear as described
It is expected the messages would respect the jenkins slack plugin configured user name and icon

Actual Results

Actual results are three messages as expected, but missing the threaded reply.
The two messages associated with a file upload are not respecting user icon or name.
Some logs exist for both the failed file upload, and for one that did not fail.

[Pipeline] writeFile
[Pipeline] slackUploadFile
[Pipeline] slackSend
Slack Send Pipeline step running, values are - baseUrl: <empty>, teamDomain: <redacted>, channel: test-builds, color: <empty>, botUser: true, tokenCredentialId: SlackBotToken, notifyCommitters: false, iconEmoji: :br3:, username: BRG Jenkins, timestamp: <empty>
[Pipeline] slackUploadFile
Using dirname=c:\jenkins\workspace\Shared\Test\TestCode and includeMask=test.txt
Adding file c:\jenkins\workspace\Shared\Test\TestCode\test.txt
ERROR: Failed uploading file to slack
[Pipeline] slackUploadFile
Using dirname=c:\jenkins\workspace\Shared\Test\TestCode and includeMask=test.txt
Adding file c:\jenkins\workspace\Shared\Test\TestCode\test.txt
Slack upload may have failed. Response: {"ok":false,"error":"channel_not_found"}
Failed to complete uploading file to Slack

Anything else?

This did not happen until we recently updated the Slack plugin. We update plugins about once a quarter. I believe prior to this I last updated July 18th/19th so my previous plugin version would have been from then.

Are you interested in contributing a fix?

No response

[timja]

Can you check this please and see if it applies:
https://github.com/jenkinsci/slack-plugin/releases/tag/711.v41060f587001

Slack deprecated the API we were using for file upload so we had to change to one that worked a bit differently.

[dclyde-brg]

Oh yes, my apologies I did forget to mention as part of debugging this I did find in the Jenkins's log that additional permissions were required.
As part of debugging this I did grant those additional permissions. That shifted the behavior slightly, as two of the three cases above were failing before I fixed the permissions.

Unfortunately the one failing case still remains in this issue.

[timja]

Is the bot invited into the channel?

Can you compare to this manifest?

https://github.com/jenkinsci/slack-plugin#creating-your-app

[dclyde-brg]

Yes, the bot is invited to the channel. As per the "Actual Results" you can see the bot (both with and without user overrides properly applying) posting successfully to the channel.

As for the manifest, I am not sure how to view my own in manifest form but the permissions applied are:

*chat:write
*chat:write.customize
*files:write
*reactions:write
+im:read
+mpim:read
*channels:read
*groups:read
*users:read
*users:read.email

Looks like I have them all (*) then two more (+). Those two more were listed in the jenkins log when I first started hitting this issue along with channels:read and groups:read so I added all four.

[timja]

see here for viewing the manifest:
https://api.slack.com/reference/manifests#updating

[dclyde-brg]

{
    "display_information": {
        "name": "Jenkins",
        "description": "Jenkins Build Service",
        "background_color": "#000060"
    },
    "features": {
        "bot_user": {
            "display_name": "Jenkins",
            "always_online": true
        }
    },
    "oauth_config": {
        "scopes": {
            "bot": [
                "chat:write",
                "chat:write.customize",
                "files:write",
                "reactions:write",
                "users:read",
                "users:read.email",
                "channels:read",
                "groups:read",
                "mpim:read",
                "im:read"
            ]
        }
    },
    "settings": {
        "org_deploy_enabled": false,
        "socket_mode_enabled": false,
        "token_rotation_enabled": false
    }
}