Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS S3 download fails when using KMS encrypted objects #21955

Open
chris-smith-zocdoc opened this issue Feb 14, 2025 · 1 comment · May be fixed by #21956
Open

AWS S3 download fails when using KMS encrypted objects #21955

chris-smith-zocdoc opened this issue Feb 14, 2025 · 1 comment · May be fixed by #21956
Labels

Comments

@chris-smith-zocdoc
Copy link
Contributor

Describe the bug

When using KMS encrypted objects, S3 requires the use of sigv4 credential signing

To reproduce, upload an object using a customer managed kms key

aws s3 cp foo.txt s3://bucket/foo.txt --sse aws:kms --sse-kms-key-id <key_id>

and try to download it using a file source

file(
    name="s3_file",
    source=http_source(
        url="s3://bucket/foo.txt",
        len=123,
        sha256="sha",
    )
)

run_shell_command(
    name="list_files",
    command="ls -lah {chroot}",
    execution_dependencies=[":s3_file"],
)

pants run :list_files

Produces an error like

IntrinsicError: Client error (400) downloading file foo.txt from https://bucket.s3.amazonaws.com/foo.txt

Pants version
2.24.1

OS
Both

Additional info
toml to enable the backend

[GLOBAL]
pants_version = "2.24.1"
backend_packages = [
    "pants.backend.shell",
    "pants.backend.url_handlers.s3",
]
plugins = [
    'botocore==1.34.135'
]
@chris-smith-zocdoc
Copy link
Contributor Author

This is the relevant signing code in the existing s3 handler

signer.add_auth(http_request)

We have a fix for this in our plugin I can push shortly, my main question is do we want to drop the old signing method an only use v4?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant