Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mc mirror without --disable-multipart generate some invalid requests #4993

Open
the-glu opened this issue Jul 22, 2024 · 1 comment
Open

Comments

@the-glu
Copy link

the-glu commented Jul 22, 2024

Expected behavior

If I do a

/opt/minio/mc mirror --remove --overwrite --retry old/deeps-data new/deeps-data

without the --disable-multipart option, minio fail with the following error:

mc: <DEBUG> POST /deeps-data/data/00/a4/1c/19-0f62-453b-9975-925b3344bd52?uploads= HTTP/1.1
Host: **REDACTED**
User-Agent: MinIO (linux; amd64) minio-go/v7.0.70 mc/RELEASE.2024-06-12T14-34-03Z
Content-Length: 0
Accept-Encoding: zstd,gzip
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20240722/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-checksum-algorithm;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
Content-Type: multipart/form-data
X-Amz-Checksum-Algorithm: CRC32C
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20240722T122202Z

mc: <DEBUG> HTTP/1.1 400 Bad Request
Content-Length: 372
Accept-Ranges: bytes
Content-Type: application/xml
Date: Mon, 22 Jul 2024 12:22:02 GMT
Server: MinIO
Strict-Transport-Security: max-age=31536000
Vary: Origin
X-Amz-Id-2: 8703973134a8a51d367677607c2ed52e3cd578ceff22cc497def47c3d201d2a1
X-Amz-Request-Id: 17E48864783FA6C6
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidRequest</Code><Message>Invalid Request (request has multiple authentication types, please use one)</Message><Resource>/deeps-data/data/00/a4/1c/19-0f62-453b-9975-925b3344bd52</Resource><RequestId>17E48864783FA6C6</RequestId><HostId>8703973134a8a51d367677607c2ed52e3cd578ceff22cc497def47c3d201d2a1</HostId></Error>mc: <DEBUG> TLS Certificate found: 
mc: <DEBUG>  >> Country: US
mc: <DEBUG>  >> Organization: Let's Encrypt
mc: <DEBUG>  >> Expires: 2024-09-29 02:13:48 +0000 UTC
mc: <DEBUG> TLS Certificate found: 
mc: <DEBUG>  >> Country: US
mc: <DEBUG>  >> Organization: Internet Security Research Group
mc: <DEBUG>  >> Expires: 2027-03-12 23:59:59 +0000 UTC
mc: <DEBUG> Response Time: 1.282901ms

Steps to reproduce the behavior

Do a mc mirror with a big file that make mc use multipart uploads ?

mc --version

mc version RELEASE.2024-06-12T14-34-03Z (commit-id=e7c9a733c680fe62066d24e8718f81938b4f6606

(minio servers, both source and target: minio version RELEASE.2024-06-13T22-53-53Z)

@klauspost
Copy link
Contributor

klauspost commented Jul 23, 2024

It seems the Content-Type: multipart/form-data is triggering isRequestPostPolicySignatureV4.

Presumably this is transferred from the source. Maybe correctly, maybe incorrectly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants