From 9880af9751519dbfb3832b8c932e64db2798436d Mon Sep 17 00:00:00 2001 From: AJ Danelz Date: Thu, 31 Aug 2023 09:52:22 -0400 Subject: [PATCH] Add instructions for MSK using SASL/SCRAM (#97) * Add instructions for MSK using SASL/SCRAM * update ports * clarify plaintext and key --- src/how-tos/amazon-msk/production.md | 36 +++++++++++++++++++++------- 1 file changed, 27 insertions(+), 9 deletions(-) diff --git a/src/how-tos/amazon-msk/production.md b/src/how-tos/amazon-msk/production.md index 75ddf32c..50958900 100644 --- a/src/how-tos/amazon-msk/production.md +++ b/src/how-tos/amazon-msk/production.md @@ -38,7 +38,7 @@ Check out the [Troubleshooting](../../reference/troubleshooting/amazon-msk.md) g Before setting up internet access to your MSK Cluster, you will need the following: -* an MSK Cluster configured for TLS encrypted client access +* an MSK Cluster configured for SASL/SCRAM authentication * an VPC security group for MSK Proxy instances * an IAM security role for MSK Proxy instances * subscription to Zilla Plus (Public MSK Proxy) via AWS Marketplace @@ -59,6 +59,20 @@ Then follow the [Create MSK Cluster](../../reference/amazon-msk/create-msk-clust Name: `aklivity`\ VPC: `my-msk-cluster`\ Subnets: `my-msk-cluster-1a` `my-msk-cluster-1b` `my-msk-cluster-1c` +Access control methods: `SASL/SCRAM authentication` + +Review and create the MSK Cluster. + +When the MSK cluster is created you will need to follow the [Sign-in credentials authentication with AWS Secrets Manager](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) to associate your `AmazonMSK_*` secret to your cluster. There will be a prompt on the cluster summary page to create a new secret or associate an existing one. For the remainder of this doc we will assume the following values for this secret: + +Name: `AmazonMSK_alice`\ +Use `Plaintext` value: + +```json:no-line-numbers +{"username":"alice","password":"alice-secret"} +``` + +Encryption key: `` ::: tip This creates your MSK cluster in preparation for secure access via the internet. @@ -77,7 +91,7 @@ Description: Kafka clients and SSH access ### Inbound Rule Type: `Custom TCP`\ -Port: `9094`\ +Port: `9096`\ Source: `` ### Inbound Rule @@ -99,7 +113,7 @@ Security Group: `default` `(MSK security group)` ### Inbound Rule Type: `Custom TCP`\ -Port: `9094`\ +Port: `9096`\ Source: `Custom Security groups`: `my-msk-proxy` ::: tip @@ -193,7 +207,7 @@ Subnets: `my-msk-cluster-1a` `my-msk-cluster-1b` `my-msk-cluster-1c` #### MSK Configuration Wildcard DNS pattern [1]: `*.aklivity.[...].amazonaws.com`\ -Port number: `9094` +Port number: `9096` #### MSK Proxy Configuration @@ -203,7 +217,7 @@ Role: `aklivity-public-msk-proxy`\ Security Groups: `my-msk-proxy`\ Secrets Manager Secret ARN [3]: [``](../../reference/amazon-msk/create-server-certificate-letsencrypt.md) Public Wildcard DNS [4]: `*.example.aklivity.io`\ -Public Port: `9094`\ +Public Port: `9096`\ Key pair for SSH access [5]: `` ### Step3. Configure stack options: `(defaults)` @@ -326,14 +340,18 @@ We use a generic Kafka client here, however the setup for any Kafka client, incl With the Kaka client now installed we are ready to configure it and point it at the Public MSK Proxy. -The MSK Proxy relies on TLS so we need to create a file called `client.properties` that tells the Kafka client to use SSL as the security protocol. +The MSK Proxy relies on encrypted SASL/SCRAM so we need to create a file called `client.properties` that tells the Kafka client to use SASL_SSL as the security protocol with SCRAM-SHA-512 encryption. + +Notice we used the default username and password, but you will need to replace those with your own credentials from the `AmazonMSK_*` secret you created. ::: code-tabs#shell @tab client.properties ```toml:no-line-numbers -security.protocol=SSL +sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="alice" password="alice-secret"; +security.protocol=SASL_SSL +sasl.mechanism=SCRAM-SHA-512 ``` ::: @@ -349,7 +367,7 @@ We can now verify that the Kafka client can successfully communicate with your M If using the wildcard DNS pattern `*.example.aklivity.io`, then we use the following as TLS bootstrap server names for the Kafka client: ```text:no-line-numbers -b-1.example.aklivity.io:9094,b-2.example.aklivity.io:9094,b-3.example.aklivity.io:9094 +b-1.example.aklivity.io:9096,b-2.example.aklivity.io:9096,b-3.example.aklivity.io:9096 ``` ::: warning @@ -358,7 +376,7 @@ Replace these TLS bootstrap server names accordingly for your own custom wildcar #### Create a Topic -Use the Kafka client to create a topic called `public-proxy-test`, replacing`` **** in the command below with the TLS proxy names of your Public MSK Proxy: +Use the Kafka client to create a topic called `public-proxy-test`, replacing `` **** in the command below with the TLS proxy names of your Public MSK Proxy: ```bash:no-line-numbers bin/kafka-topics.sh --create --topic public-proxy-test --partitions 3 --replication-factor 3 --command-config client.properties --bootstrap-server