Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support custom prefixes for S3 and STS #75

Merged
merged 1 commit into from
Jun 20, 2024

Conversation

vagaerg
Copy link
Member

@vagaerg vagaerg commented Jun 20, 2024

Closes #57

@cla-bot cla-bot bot added the cla-signed label Jun 20, 2024
@Config("s3proxy.hostname")
@ConfigDescription("Hostname to use for REST operations, virtual-host style addressing is only supported if this is set")
@Config("s3proxy.s3.hostname")
@ConfigDescription("Hostname to use for S3 REST operations, virtual-host style addressing is only supported if this is set")
public TrinoS3ProxyConfig setHostName(String hostName)
{
this.hostName = Optional.ofNullable(hostName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's rename this (and the method) s3Hostname

public static final String S3_PATH = BASE_PATH + "s3";
public static final String STS_PATH = BASE_PATH + "sts";
private static final String BASE_PATH = "/api/v1/s3Proxy/";
public static final String DEFAULT_S3_PATH = BASE_PATH + "s3";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These constants aren't really useful any more. I'd inline them. Let's make BASE_PATH a config as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was that S3_PATH and STS_PATH are now fully independent. BASE_PATH is only used here for the default.

So someone may have s3proxy.s3.prefix=/s3 and s3proxy.sts.prefix=/sts and there's no common part to them

Agreed on removing the constants

@vagaerg vagaerg force-pushed the configurable-paths branch from 4ebd242 to e34a0df Compare June 20, 2024 17:26
@@ -52,11 +53,16 @@ public class TrinoS3ProxyServerModule
@Override
protected void setup(Binder binder)
{
configBinder(binder).bindConfig(SigningControllerConfig.class);
configBinder(binder).bindConfig(TrinoS3ProxyConfig.class);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildConfigObject already does the binding so this is redundant


@Config("s3proxy.s3.prefix")
@ConfigDescription("URL Prefix for S3 operations, optional")
public TrinoS3ProxyConfig setS3Prefix(String s3Prefix)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is prefix the right noun? This is the full path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By prefix I meant "all S3 operations will be under this path" - meaning all S3 URLs will be this path or something nested within it.

Happy to change it to s3Path if you think that's more appropriate but I thought prefix was reasonable

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Path is better

@vagaerg vagaerg force-pushed the configurable-paths branch from e34a0df to 54ae608 Compare June 20, 2024 17:59
@@ -45,7 +44,7 @@ public TrinoS3ProxyResource(SigningController signingController, TrinoS3ProxyCli
{
this.signingController = requireNonNull(signingController, "signingController is null");
this.proxyClient = requireNonNull(proxyClient, "proxyClient is null");
this.serverHostName = requireNonNull(trinoS3ProxyConfig, "restConfig is null").getHostName();
this.serverHostName = requireNonNull(trinoS3ProxyConfig, "trinoS3ProxyConfig is null").getS3HostName();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't usually check for null on any object we're using - i.e. config objects. You'd get an early NPE anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was debating this one, thanks for clarifying

@vagaerg vagaerg force-pushed the configurable-paths branch from 54ae608 to a46893d Compare June 20, 2024 18:20
@@ -62,8 +60,8 @@ public Object createTestInstance(TestInstanceFactoryContext factoryContext, Exte

TestingTrinoS3ProxyServer trinoS3ProxyServer = builder
.withMockS3Container()
.addModule(binder -> binder.bind(TestingS3ClientProvider.TestingS3ClientConfig.class).in(Scopes.SINGLETON))
Copy link
Member

@Randgalt Randgalt Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be in the child binder where TestingS3ClientProvider is bound

@vagaerg vagaerg force-pushed the configurable-paths branch from a46893d to ffe546e Compare June 20, 2024 18:24
Copy link
Member

@Randgalt Randgalt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor things to address

@vagaerg vagaerg force-pushed the configurable-paths branch from ffe546e to 2ec2f45 Compare June 20, 2024 18:26
@vagaerg vagaerg merged commit 406ddc0 into trinodb:main Jun 20, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Customisable API path
2 participants