How can I use attribute-based access control (ABAC) in pipelines?

[bolodecenouracomcafe]

Hello everyone,

Here is the scenario in my multi-tenant application on AWS OpenSearch Service:

• One domain/cluster for all tenants.
• One index per tenant.

To ensure tenant isolation, it is recommended to have one data prepper pipeline per tenant.

In the sink configuration, there is a field called sts_role_arn. This allows me to define a role that restricts access to only the tenant's index.

However, I have a problem: I can't create a separate role for each tenant because there is a limit on the number of roles (and we have thousands of tenants). Instead, the solution is to use attribute-based access control (ABAC), as mentioned in this article.

In my application, when retrieving data from the tenant index, I use temporary credentials (assume role) with ABAC.

Here’s how it works:

We created a single role with access to all indexes.

When I assume the role, I dynamically create a policy to restrict access to the specific tenant's index.

Role Definition:

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": "es:*",
         "Resource": "arn:aws:es:Region:account-id:domain/test-domain/*"
      }
   ]
}

Dynamic Policy Generation:

var policy = """
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": "es:*",
         "Resource": "arn:aws:es:Region:account-id:domain/test-domain/index_${idTenant}"
      }
   ]
}
""".replace("${idTenant}", context.getIdTenant().toString());

try (StsClient stsClient = StsClient.create()) {
    var builder = AssumeRoleRequest.builder();
    builder.roleArn(this.getRoleArn(XXXXXXX));
    builder.roleSessionName(YYYYYY);
    builder.policy(policy);

    AssumeRoleResponse assumeRoleResponse = stsClient.assumeRole(builder.build());
    return assumeRoleResponse.credentials();
} catch (Exception e) {
    throw new RuntimeException("Problem connecting to STS. Message: " + e.getMessage(), e);
}

Now, I need similar behavior for creating a sink. I can use the sts_role_arn field, but I also need a way to define a complementary policy dynamically to guarantee tenant isolation.

I will need this on S3 as a source as well.

[graytaylor0]

If the same role is used to send to OpenSearch Service across different pipelines, I am not aware of a way to only allow the pipeline to send to a specific index. Maybe there is some path for Data Prepper to be configurable to block calls to any other index by dynamically applying a policy to the assume role call like you're suggestion, but using the same role as is for both the domain access policy and FGAC permissions will provide permissions the full permissions to the role (https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html, https://opensearch.org/docs/latest/security/access-control/permissions/#index-permissions).

[bolodecenouracomcafe]

Sorry for the delay, I was validating the use of FGAC as a solution.

So, I think that FGAC is a valid option. I need to create an OpenSearch internal user per tenant and give permission only for their own index. Each tenant has a user and password, so I avoid a cross-tenant incident.

I verified that I can't use an IAM user as an OpenSearch user because I would have the same problem: I would need to have an IAM role per tenant.

About using S3 as a source, I didn't find an option for better tenant isolation. I have to define a role that can access the entire bucket and filter by the prefix. This is bad because it is a critical feature, and I would like to have as much isolation as possible.

Do you have any suggestion to increase the isolation when using S3 as a source in a multi-tenant scenario?