Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MIDRC-901 Do not copy all original headers #37

Merged
merged 3 commits into from
Dec 16, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 11 additions & 18 deletions gen3workflow/routes/s3.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,24 +109,17 @@ async def s3_endpoint(path: str, request: Request):
region = config["USER_BUCKETS_REGION"]
service = "s3"

# generate the request headers:
# - first, copy all the headers from the original request.
headers = dict(request.headers)
# - remove the `authorization` header: it contains a Gen3 token instead of an AWS IAM key.
# The new `authorization` header will be added _after_ generating the signature.
headers.pop("authorization")
# - overwrite the `x-amz-content-sha256` header value with the body hash. When this header is
# set to "STREAMING-AWS4-HMAC-SHA256-PAYLOAD" in the original request (payload sent over
# multiple chunks), we replace it with the body hash (because I couldn't get the signing to
# work for "STREAMING-AWS4-HMAC-SHA256-PAYLOAD" - I believe it requires using the signature
# from the previous chunk).
# NOTE: This may cause issues when large files are _actually_ uploaded over multiple chunks.
headers["x-amz-content-sha256"] = body_hash
# - remove the `content-md5` header: when the `x-amz-content-sha256` header is overwritten (see
# above), the original `content-md5` value becomes incorrect. It's not required in V4 signing.
headers.pop("content-md5", None)
# - replace the `host` header, since we are re-signing and sending to a different host.
headers["host"] = f"{user_bucket}.s3.amazonaws.com"
# generate the request headers.
# overwrite the original `x-amz-content-sha256` header value with the body hash. When this
# header is set to "STREAMING-AWS4-HMAC-SHA256-PAYLOAD" in the original request (payload sent
# over multiple chunks), we still replace it with the body hash (because I couldn't get the
# signing to work for "STREAMING-AWS4-HMAC-SHA256-PAYLOAD" - I believe it requires using the signature from the previous chunk).
# NOTE: This may cause issues when large files are _actually_ uploaded over multiple chunks.
headers = {
"host": f"{user_bucket}.s3.amazonaws.com",
"x-amz-content-sha256": body_hash,
"x-amz-date": timestamp,
}

# get AWS credentials from the configuration or the current assumed role session
if config["S3_ENDPOINTS_AWS_ACCESS_KEY_ID"]:
Expand Down
Loading