Skip to content

Commit

Permalink
Add instructions for MSK using SASL/SCRAM (#97)
Browse files Browse the repository at this point in the history
* Add instructions for MSK using SASL/SCRAM

* update ports

* clarify plaintext and key
  • Loading branch information
vordimous authored Aug 31, 2023
1 parent e462853 commit 9880af9
Showing 1 changed file with 27 additions and 9 deletions.
36 changes: 27 additions & 9 deletions src/how-tos/amazon-msk/production.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Check out the [Troubleshooting](../../reference/troubleshooting/amazon-msk.md) g

Before setting up internet access to your MSK Cluster, you will need the following:

* an MSK Cluster configured for TLS encrypted client access
* an MSK Cluster configured for SASL/SCRAM authentication
* an VPC security group for MSK Proxy instances
* an IAM security role for MSK Proxy instances
* subscription to Zilla Plus (Public MSK Proxy) via AWS Marketplace
Expand All @@ -59,6 +59,20 @@ Then follow the [Create MSK Cluster](../../reference/amazon-msk/create-msk-clust
Name: `aklivity`\
VPC: `my-msk-cluster`\
Subnets: `my-msk-cluster-1a` `my-msk-cluster-1b` `my-msk-cluster-1c`
Access control methods: `SASL/SCRAM authentication`

Review and create the MSK Cluster.

When the MSK cluster is created you will need to follow the [Sign-in credentials authentication with AWS Secrets Manager](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) to associate your `AmazonMSK_*` secret to your cluster. There will be a prompt on the cluster summary page to create a new secret or associate an existing one. For the remainder of this doc we will assume the following values for this secret:

Name: `AmazonMSK_alice`\
Use `Plaintext` value:

```json:no-line-numbers
{"username":"alice","password":"alice-secret"}
```

Encryption key: `<Customer managed key>`

::: tip
This creates your MSK cluster in preparation for secure access via the internet.
Expand All @@ -77,7 +91,7 @@ Description: Kafka clients and SSH access
### Inbound Rule

Type: `Custom TCP`\
Port: `9094`\
Port: `9096`\
Source: `<Any IPv4>`

### Inbound Rule
Expand All @@ -99,7 +113,7 @@ Security Group: `default` `(MSK security group)`
### Inbound Rule

Type: `Custom TCP`\
Port: `9094`\
Port: `9096`\
Source: `Custom Security groups`: `my-msk-proxy`

::: tip
Expand Down Expand Up @@ -193,7 +207,7 @@ Subnets: `my-msk-cluster-1a` `my-msk-cluster-1b` `my-msk-cluster-1c`
#### MSK Configuration

Wildcard DNS pattern [1]: `*.aklivity.[...].amazonaws.com`\
Port number: `9094`
Port number: `9096`

#### MSK Proxy Configuration

Expand All @@ -203,7 +217,7 @@ Role: `aklivity-public-msk-proxy`\
Security Groups: `my-msk-proxy`\
Secrets Manager Secret ARN [3]: [`<LetsEncrypt signed certificate's private key secret ARN>`](../../reference/amazon-msk/create-server-certificate-letsencrypt.md)
Public Wildcard DNS [4]: `*.example.aklivity.io`\
Public Port: `9094`\
Public Port: `9096`\
Key pair for SSH access [5]: `<key pair>`

### Step3. Configure stack options: `(defaults)`
Expand Down Expand Up @@ -326,14 +340,18 @@ We use a generic Kafka client here, however the setup for any Kafka client, incl

With the Kaka client now installed we are ready to configure it and point it at the Public MSK Proxy.

The MSK Proxy relies on TLS so we need to create a file called `client.properties` that tells the Kafka client to use SSL as the security protocol.
The MSK Proxy relies on encrypted SASL/SCRAM so we need to create a file called `client.properties` that tells the Kafka client to use SASL_SSL as the security protocol with SCRAM-SHA-512 encryption.

Notice we used the default username and password, but you will need to replace those with your own credentials from the `AmazonMSK_*` secret you created.

::: code-tabs#shell

@tab client.properties

```toml:no-line-numbers
security.protocol=SSL
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="alice" password="alice-secret";
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
```

:::
Expand All @@ -349,7 +367,7 @@ We can now verify that the Kafka client can successfully communicate with your M
If using the wildcard DNS pattern `*.example.aklivity.io`, then we use the following as TLS bootstrap server names for the Kafka client:

```text:no-line-numbers
b-1.example.aklivity.io:9094,b-2.example.aklivity.io:9094,b-3.example.aklivity.io:9094
b-1.example.aklivity.io:9096,b-2.example.aklivity.io:9096,b-3.example.aklivity.io:9096
```

::: warning
Expand All @@ -358,7 +376,7 @@ Replace these TLS bootstrap server names accordingly for your own custom wildcar

#### Create a Topic

Use the Kafka client to create a topic called `public-proxy-test`, replacing`<tls-bootstrap-server-names>` **** in the command below with the TLS proxy names of your Public MSK Proxy:
Use the Kafka client to create a topic called `public-proxy-test`, replacing `<tls-bootstrap-server-names>` **** in the command below with the TLS proxy names of your Public MSK Proxy:

```bash:no-line-numbers
bin/kafka-topics.sh --create --topic public-proxy-test --partitions 3 --replication-factor 3 --command-config client.properties --bootstrap-server <tls-bootstrap-server-names>
Expand Down

0 comments on commit 9880af9

Please sign in to comment.