From 5988a8696bacb3ecbbcf76eeda986c4f60122260 Mon Sep 17 00:00:00 2001 From: AJ Date: Wed, 30 Aug 2023 09:17:52 -0400 Subject: [PATCH 1/3] Add instructions for MSK using SASL/SCRAM --- src/how-tos/amazon-msk/production.md | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/src/how-tos/amazon-msk/production.md b/src/how-tos/amazon-msk/production.md index 75ddf32c..9308f201 100644 --- a/src/how-tos/amazon-msk/production.md +++ b/src/how-tos/amazon-msk/production.md @@ -38,7 +38,7 @@ Check out the [Troubleshooting](../../reference/troubleshooting/amazon-msk.md) g Before setting up internet access to your MSK Cluster, you will need the following: -* an MSK Cluster configured for TLS encrypted client access +* an MSK Cluster configured for SASL/SCRAM authentication * an VPC security group for MSK Proxy instances * an IAM security role for MSK Proxy instances * subscription to Zilla Plus (Public MSK Proxy) via AWS Marketplace @@ -59,6 +59,15 @@ Then follow the [Create MSK Cluster](../../reference/amazon-msk/create-msk-clust Name: `aklivity`\ VPC: `my-msk-cluster`\ Subnets: `my-msk-cluster-1a` `my-msk-cluster-1b` `my-msk-cluster-1c` +Access control methods: `SASL/SCRAM authentication` + +Review and create the MSK Cluster. + +When the MSK cluster is created you will need to follow the [Sign-in credentials authentication with AWS Secrets Manager](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) to associate your `AmazonMSK_*` secret to your cluster. For the remainder of this doc we will assume the following values for this secret: + +Name: `AmazonMSK_alice`\ +Username: `alice`\ +Password: `alice-secret` ::: tip This creates your MSK cluster in preparation for secure access via the internet. @@ -77,7 +86,7 @@ Description: Kafka clients and SSH access ### Inbound Rule Type: `Custom TCP`\ -Port: `9094`\ +Port: `9096`\ Source: `` ### Inbound Rule @@ -99,7 +108,7 @@ Security Group: `default` `(MSK security group)` ### Inbound Rule Type: `Custom TCP`\ -Port: `9094`\ +Port: `9096`\ Source: `Custom Security groups`: `my-msk-proxy` ::: tip @@ -326,14 +335,18 @@ We use a generic Kafka client here, however the setup for any Kafka client, incl With the Kaka client now installed we are ready to configure it and point it at the Public MSK Proxy. -The MSK Proxy relies on TLS so we need to create a file called `client.properties` that tells the Kafka client to use SSL as the security protocol. +The MSK Proxy relies on encrypted SASL/SCRAM so we need to create a file called `client.properties` that tells the Kafka client to use SASL_SSL as the security protocol with SCRAM-SHA-512 encryption. + +Notice we used the default username and password, but you will need to replace those with your own credentials from the `AmazonMSK_*` secret you created. ::: code-tabs#shell @tab client.properties ```toml:no-line-numbers -security.protocol=SSL +sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="alice" password="alice-secret"; +security.protocol=SASL_SSL +sasl.mechanism=SCRAM-SHA-512 ``` ::: @@ -358,7 +371,7 @@ Replace these TLS bootstrap server names accordingly for your own custom wildcar #### Create a Topic -Use the Kafka client to create a topic called `public-proxy-test`, replacing`` **** in the command below with the TLS proxy names of your Public MSK Proxy: +Use the Kafka client to create a topic called `public-proxy-test`, replacing `` **** in the command below with the TLS proxy names of your Public MSK Proxy: ```bash:no-line-numbers bin/kafka-topics.sh --create --topic public-proxy-test --partitions 3 --replication-factor 3 --command-config client.properties --bootstrap-server From 68604b7abbb6a153f0e8abcd941ecd933ea47942 Mon Sep 17 00:00:00 2001 From: AJ Date: Wed, 30 Aug 2023 10:36:06 -0400 Subject: [PATCH 2/3] update ports --- src/how-tos/amazon-msk/production.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/how-tos/amazon-msk/production.md b/src/how-tos/amazon-msk/production.md index 9308f201..35893f0d 100644 --- a/src/how-tos/amazon-msk/production.md +++ b/src/how-tos/amazon-msk/production.md @@ -202,7 +202,7 @@ Subnets: `my-msk-cluster-1a` `my-msk-cluster-1b` `my-msk-cluster-1c` #### MSK Configuration Wildcard DNS pattern [1]: `*.aklivity.[...].amazonaws.com`\ -Port number: `9094` +Port number: `9096` #### MSK Proxy Configuration @@ -212,7 +212,7 @@ Role: `aklivity-public-msk-proxy`\ Security Groups: `my-msk-proxy`\ Secrets Manager Secret ARN [3]: [``](../../reference/amazon-msk/create-server-certificate-letsencrypt.md) Public Wildcard DNS [4]: `*.example.aklivity.io`\ -Public Port: `9094`\ +Public Port: `9096`\ Key pair for SSH access [5]: `` ### Step3. Configure stack options: `(defaults)` @@ -362,7 +362,7 @@ We can now verify that the Kafka client can successfully communicate with your M If using the wildcard DNS pattern `*.example.aklivity.io`, then we use the following as TLS bootstrap server names for the Kafka client: ```text:no-line-numbers -b-1.example.aklivity.io:9094,b-2.example.aklivity.io:9094,b-3.example.aklivity.io:9094 +b-1.example.aklivity.io:9096,b-2.example.aklivity.io:9096,b-3.example.aklivity.io:9096 ``` ::: warning From 080200a719c2abba34ad2b6d1581fb69bfef1589 Mon Sep 17 00:00:00 2001 From: AJ Date: Wed, 30 Aug 2023 10:58:23 -0400 Subject: [PATCH 3/3] clarify plaintext and key --- src/how-tos/amazon-msk/production.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/how-tos/amazon-msk/production.md b/src/how-tos/amazon-msk/production.md index 35893f0d..50958900 100644 --- a/src/how-tos/amazon-msk/production.md +++ b/src/how-tos/amazon-msk/production.md @@ -63,11 +63,16 @@ Access control methods: `SASL/SCRAM authentication` Review and create the MSK Cluster. -When the MSK cluster is created you will need to follow the [Sign-in credentials authentication with AWS Secrets Manager](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) to associate your `AmazonMSK_*` secret to your cluster. For the remainder of this doc we will assume the following values for this secret: +When the MSK cluster is created you will need to follow the [Sign-in credentials authentication with AWS Secrets Manager](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) to associate your `AmazonMSK_*` secret to your cluster. There will be a prompt on the cluster summary page to create a new secret or associate an existing one. For the remainder of this doc we will assume the following values for this secret: Name: `AmazonMSK_alice`\ -Username: `alice`\ -Password: `alice-secret` +Use `Plaintext` value: + +```json:no-line-numbers +{"username":"alice","password":"alice-secret"} +``` + +Encryption key: `` ::: tip This creates your MSK cluster in preparation for secure access via the internet.