SigV4: Allow specifying Host header as config #134
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of determining the Host header value from url.getHost(), allow specifying the Host to use for the SigV4 signature calculation via config. Rationale: - Assume you're using the Gerrit LFS plugin and pointing it to AWS S3 or GCS (S3 REST API) - Your Gerrit (and git clients pulling/cloning) are hosted on an on-prem k8s cluster - You download say 40TB/month of LFS content (because your CI system needs it maybe) - The network egress costs from AWS S3 / GCS for pulling all this content will easily be $2-5k+/ month - So you will want to maintain an on-prem caching proxy e.g. NGINX, Varnish etc - That is, you will set hostname in your lfs.config on Gerrit to my-varnish-proxy.internaldomain.io - my-varnish-proxy will proxy the requests to storage.googleapis.com - In order to avoid adding SigV4 logic in your proxy, it is necessary that the Gerrit LFS layer uses the S3 endpoint to calculate the SigV4 signature - This is because when GCP is verifying the signature in the presigned URL, it seems to be verifying against the Host: storage.googleapis.com and not the Host header passed in/overridden via the proxy MinIO used to support this exact use case. It provided its own S3 API (with SigV4 auth), and then it proxied the requests to GCS. However, as of 2022 MinIO deprecated support for proxying to GCS [1] [1] https://blog.min.io/deprecation-of-the-minio-gateway/ Change-Id: Id2df956c50d43b83ffec3c7955dc34504cde616b
|
I know JGit does code reviews in the eclipse GerritHub. I was not quickly able to set up an account on that Gerrit. This diff is a test diff to get some preliminary feedback on the community's willingness to accept this patch. |
|
Follow the contributor guide for instructions how to push changes for review to GerritHub |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of determining the Host header value from url.getHost(), allow specifying the Host to use for the SigV4 signature calculation via config.
Rationale:
MinIO used to support this exact use case. It provided its own S3 API (with SigV4 auth), and then it proxied the requests to GCS. However, as of 2022 MinIO deprecated support for proxying to GCS [1]
[1] https://blog.min.io/deprecation-of-the-minio-gateway/
Change-Id: Id2df956c50d43b83ffec3c7955dc34504cde616b