Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add a reference to the external CDKTF repo #247

Merged
merged 5 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"autoscale",
"Bidi",
"bitnami",
"CDKTF",
"cleartext",
"Clickstream",
"confluentinc",
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"name": "zilla-docs",
"type": "module",
"version": "2.1.0",
"description": "The official documentation for the aklivity/zilla open-source project",
"keywords": [],
Expand Down
51 changes: 40 additions & 11 deletions src/.vuepress/sidebar/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ export const enSidebar = sidebar({
{
text: "Events",
link: "events.md",
children: [],
},
{
text: "Metrics",
Expand Down Expand Up @@ -86,10 +85,12 @@ export const enSidebar = sidebar({
{
text: "CLI (zpm)",
link: "zpm-cli.md",
children: [],
},
{
text: "zpm Config Overview",
link: "overview.md",
children: [],
},
],
},
Expand All @@ -102,6 +103,7 @@ export const enSidebar = sidebar({
{
text: "Overview",
link: "concepts/kafka-proxies/secure-public-access.md",
children: [],
},
{
text: "Amazon MSK",
Expand All @@ -110,26 +112,38 @@ export const enSidebar = sidebar({
link: "how-tos/amazon-msk/secure-public-access/overview.md",
children: [
{
text: "Overview",
text: "Deployment Options",
link: "overview.md",
},
{
text: "via SASL/SCRAM",
link: "production.md",
},
{
text: "via mTLS",
link: "production-mutual-tls.md",
text: "Terraform",
link: "https://github.com/aklivity/zilla-plus-aws-templates/tree/main/amazon-msk/cdktf/secure-public-access",
},
{
text: "via Unauthorized access",
link: "development.md",
text: "CloudFormation",
collapsible: true,
link: "production.md",
children: [
{
text: "with SASL/SCRAM",
link: "production.md",
},
{
text: "with Mutual TLS (mTLS)",
link: "production-mutual-tls.md",
},
{
text: "with Unauthorized access",
link: "development.md",
},
],
},
],
},
{
text: "Confluent Cloud",
link: "how-tos/confluent-cloud/secure-public-access.md",
children: [],
},
],
},
Expand All @@ -140,18 +154,22 @@ export const enSidebar = sidebar({
{
text: "Overview",
link: "concepts/kafka-proxies/iot-ingest-control.md",
children: [],
},
{
text: "Amazon MSK",
link: "how-tos/amazon-msk/iot-ingest-control.md",
children: [],
},
{
text: "Confluent Cloud",
link: "how-tos/confluent-cloud/iot-ingest-control.md",
children: [],
},
{
text: "Redpanda",
link: "how-tos/redpanda/iot-ingest-control.md",
children: [],
},
],
},
Expand All @@ -161,7 +179,6 @@ export const enSidebar = sidebar({
{
text: "AWS",
prefix: "how-tos/aws-services/",
link: "how-tos/aws-services/",
children: "structure",
},
],
Expand Down Expand Up @@ -219,6 +236,12 @@ export const enSidebar = sidebar({
{
text: "Kafka Proxying",
children: [
{
text: "-hidden-",
ariaLabel: "-hidden-",
link: "/",
children: [],
},
{
text: "HTTP Kafka Proxy",
collapsible: true,
Expand Down Expand Up @@ -326,6 +349,12 @@ export const enSidebar = sidebar({
{
text: "Security",
children: [
{
text: "-hidden-",
ariaLabel: "-hidden-",
link: "/",
children: [],
},
{
text: "Guards",
collapsible: true,
Expand Down
13 changes: 11 additions & 2 deletions src/.vuepress/styles/index.scss
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,12 @@ h6 {
}
}

// TODO: This is a hack to be removed along with the corresponding sidebar elements
// when the bug (https://github.com/vuepress-theme-hope/vuepress-theme-hope/issues/4286) is fixed
.vp-sidebar-header:has([aria-label^="-hidden-"]) {
display: none !important;
}

::-webkit-scrollbar-thumb {
background: var(--code-highlight-bg-color);
}
Expand All @@ -132,16 +138,19 @@ h6 {
.vp-sidebar-group > .vp-sidebar-header {
padding: 0.25rem 0rem;
margin-left: 0rem;
font-weight: 600;
}
.vp-sidebar-links {
.vp-sidebar-link {
padding: 0.25rem 0rem;
padding: 0.25rem 0.5rem;
margin-left: 0rem;
}
.vp-sidebar-links {
.vp-sidebar-group {
padding: 0.25rem 0.5rem;
}
.vp-sidebar-link {
padding: 0.25rem 0.5rem;
margin: 0rem 0.5;
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions src/how-tos/connecting-to-kafka/amazon-msk.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ MSK's “Public Access” feature directly exposes your brokers to the internet,
Once your MSK cluster is reachable over the internet, it will rely on `mTLS` to authenticate external clients. Setting up `mTLS` between MSK and Zilla is done in three steps:

1. Create a trusted Client Certificate in Amazon Certificate Manager.
2. Export the Client Certificate as well as the Certificate Authority (CA) Certificate.
3. Create a PKCS12 KeyStore containing the exported certificates that will be referenced by Zilla to complete the `mTLS` handshake with your MSK cluster.
1. Export the Client Certificate as well as the Certificate Authority (CA) Certificate.
1. Create a PKCS12 KeyStore containing the exported certificates that will be referenced by Zilla to complete the `mTLS` handshake with your MSK cluster.

::: info NOTE
If you deployed the Zilla Plus for Amazon MSK, then you should already have a Client Certificate that Zilla can use and you can go straight to the second step.
Expand Down
4 changes: 2 additions & 2 deletions src/solutions/concepts/kafka-proxies/iot-ingest-control.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ You will need to choose a wildcard DNS pattern to use for public internet access

## Confluent Cloud

The [Zilla Plus for Confluent Cloud](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) IoT Ingest and Control Broker lets authorized Kafka clients connect, publish messages and subscribe to topics in your Confluent Cloud cluster via the internet.
The [Zilla Plus for Confluent Cloud](https://aws.amazon.com/marketplace/pp/prodview-eblxkinsqbaks) IoT Ingest and Control Broker lets authorized Kafka clients connect, publish messages and subscribe to topics in your Confluent Cloud cluster via the internet.

> [Follow the guide to get started](../../how-tos/confluent-cloud/iot-ingest-control.md)
## Redpanda

The [Zilla Plus for Redpanda](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) IoT Ingest and Control Broker lets authorized Kafka clients connect, publish messages and subscribe to topics in your Redpanda cluster via the internet.
The [Zilla Plus for Redpanda](https://aws.amazon.com/marketplace/pp/prodview-sj4kquyndubiu) IoT Ingest and Control Broker lets authorized Kafka clients connect, publish messages and subscribe to topics in your Redpanda cluster via the internet.

> [Follow the guide to get started](../../how-tos/redpanda/iot-ingest-control.md)
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ The [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-j
## Confluent Cloud

The [Zilla Plus for Confluent Cloud](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) Secure Public Access proxy lets authorized Kafka clients connect, publish messages and subscribe to topics in your Confluent Cloud cluster via the internet.
The [Zilla Plus for Confluent Cloud](https://aws.amazon.com/marketplace/pp/prodview-eblxkinsqbaks) Secure Public Access proxy lets authorized Kafka clients connect, publish messages and subscribe to topics in your Confluent Cloud cluster via the internet.

> [Follow the guide to get started](../../how-tos/confluent-cloud/secure-public-access.md)
20 changes: 20 additions & 0 deletions src/solutions/how-tos/amazon-msk/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
icon: aky-zilla-plus
description: Zilla Plus for Amazon MSK, you can create publicly reachable Kafka endpoints into an Amazon MSK cluster. You can also expose topics inside your Amazon MSK cluster via declaratively defined REST, SSE, gRPC, and MQTT APIs.
---

# Zilla Plus for Amazon MSK

With [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44), you can create publicly reachable Kafka endpoints into an Amazon MSK cluster. You can also expose topics inside your Amazon MSK cluster via declaratively defined REST, SSE, gRPC, and MQTT APIs.

## IoT Ingest and Control

The IoT Ingest and Control MQTT Broker that lets clients publish messages and subscribe to topics proxied to Kafka topics in your Amazon MSK cluster.

> [Amazon MSK IoT Ingest and Control Guide](./iot-ingest-control.md)

## Secure Public Access

The Secure Public Access Proxy lets authorized Kafka clients connect, publish messages and subscribe to topics in your Amazon MSK cluster via the internet.

> [Amazon MSK Secure Public Access Proxy Guide](./secure-public-access/overview.md)
8 changes: 4 additions & 4 deletions src/solutions/how-tos/amazon-msk/iot-ingest-control.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
icon: aky-zilla-plus
description: Set up an IoT Ingest and Control MQTT Broker that lets clients publish messages and subscribe to topics proxied to Kafka topics in your Amazon MSK cluster.
description: The IoT Ingest and Control MQTT Broker that lets clients publish messages and subscribe to topics proxied to Kafka topics in your Amazon MSK cluster.
---

# Amazon MSK IoT Ingest and Control
Expand Down Expand Up @@ -79,9 +79,9 @@ Go to the newly created secret and under `Resource permissions` > select `Edit P

## Subscribe via AWS Marketplace

The [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-sj4kquyndubiu) is available through the AWS Marketplace. You can skip this step if you have already subscribed to Zilla Plus for Amazon MSK via the AWS Marketplace.
The [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44)) is available through the AWS Marketplace. You can skip this step if you have already subscribed to Zilla Plus for Amazon MSK via the AWS Marketplace.

To get started, visit the Proxy's Marketplace [Product Page](https://aws.amazon.com/marketplace/pp/prodview-sj4kquyndubiu) and `Subscribe` to the offering. You should now see `Zilla Plus for Amazon MSK` listed in your [AWS Marketplace](https://console.aws.amazon.com/marketplace) subscriptions.
To get started, visit the Proxy's Marketplace [Product Page](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) and `Subscribe` to the offering. You should now see `Zilla Plus for Amazon MSK` listed in your [AWS Marketplace](https://console.aws.amazon.com/marketplace) subscriptions.

## Deploy the IoT Ingest and Control MQTT Broker

Expand Down Expand Up @@ -147,4 +147,4 @@ Parameters:

## Conclusion

You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-sj4kquyndubiu) IoT Ingest and Control MQTT Broker. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../aws-services/manage-cloudformation-stack.md) section.
You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) IoT Ingest and Control MQTT Broker. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../aws-services/manage-cloudformation-stack.md) section.
Original file line number Diff line number Diff line change
Expand Up @@ -486,16 +486,16 @@ b-1.aklivity.example.com:9094,b-2.aklivity.example.com:9094,b-3.aklivity.example
::: tip A quick summary of what just happened

1. The Kafka client with access to the public internet issued a request to create a new topic
2. This request was directed to the internet-facing Network Load Balancer
3. The Network Load Balancer forwarded the request to the <ZillaPlus/> proxy
4. The <ZillaPlus/> proxy routed the request to the appropriate MSK broker
5. The topic was created in the MSK broker
6. Public access was verified
1. This request was directed to the internet-facing Network Load Balancer
1. The Network Load Balancer forwarded the request to the <ZillaPlus/> proxy
1. The <ZillaPlus/> proxy routed the request to the appropriate MSK broker
1. The topic was created in the MSK broker
1. Public access was verified

:::

<!-- @include: @partials/secure-public-access/send-message.md -->

## Conclusion

You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-sj4kquyndubiu) Secure Public Access. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../../aws-services/manage-cloudformation-stack.md) section.
You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) Secure Public Access. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../../aws-services/manage-cloudformation-stack.md) section.
19 changes: 11 additions & 8 deletions src/solutions/how-tos/amazon-msk/secure-public-access/overview.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
---
icon: aky-zilla-plus
description: Securely access your Amazon MSK cluster via the internet.
description: The Secure Public Access Proxy lets authorized Kafka clients connect, publish messages and subscribe to topics in your Amazon MSK cluster via the internet.
---

# Amazon MSK Secure Public Access Proxy

[Available in <ZillaPlus/>](https://www.aklivity.io/products/zilla-plus)
{.zilla-plus-badge .hint-container .info}

::: info Estimated time to complete 20-30 minutes.
:::

The [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) Secure Public Access Proxy lets authorized Kafka clients connect, publish messages and subscribe to topics in your Amazon MSK cluster via the internet.

By automating the configuration of an internet-facing network load balancer and auto-scaling group of stateless Secure Public Access proxies to expose your MSK cluster via the public internet, Kafka clients can connect, publish messages and subscribe to topics in your Amazon MSK cluster from outside AWS.
Expand All @@ -19,14 +16,20 @@ You will need to choose a wildcard DNS pattern to use for public internet access

The <ZillaPlus/> proxy can securely expose any MSK cluster with these access options.

## SASL/SCRAM authentication
## Deployment with Terraform

Follow the [Secure Public Access with Terraform](https://github.com/aklivity/zilla-plus-aws-templates/tree/main/amazon-msk/cdktf/secure-public-access) guide to generated or deploy a custom Terraform template using [CDKTF](https://developer.hashicorp.com/terraform/cdktf). This Terraform script can be configured to deploy `SASL/SCRAM authentication`, `Mutual TLS (mTLS) authentication` or `Unauthorized access` to setup connectivity to your MSK cluster with a wildcard DNS pattern.

## Deployment with CloudFormation

### SASL/SCRAM authentication

Follow the [Secure Public Access via SASL/SCRAM authentication](./production.md) guide to setup connectivity to your MSK cluster using a globally trusted TLS server certificate with a wildcard DNS pattern `*.example.aklivity.io` to illustrate the steps.

## Mutual TLS (mTLS) authentication
### Mutual TLS (mTLS) authentication

Follow the [Secure Public Access via mTLS](./production-mutual-tls.md) guide to setup connectivity to your MSK cluster using a globally trusted TLS server certificate with a wildcard DNS pattern `*.example.aklivity.io` to illustrate the steps.
Follow the [Secure Public Access via mTLS](./production-mutual-tls.md) guide to setup connectivity to your MSK cluster using a globally trusted TLS server certificate with a wildcard DNS pattern `*.example.aklivity.io` to illustrate the steps.

## Unauthorized access
### Unauthorized access

Follow the [Secure Public Access via Unauthorized access](./development.md) guide to setup connectivity to your MSK cluster using a locally trusted TLS server certificate with the example wildcard DNS pattern `*.aklivity.example.com`.
Original file line number Diff line number Diff line change
Expand Up @@ -476,18 +476,18 @@ Replace these TLS bootstrap server names accordingly for your own custom wildcar
::: tip A quick summary of what just happened

1. The Kafka client with access to the public internet issued a request to create a new topic
2. This request was directed to the internet-facing Network Load Balancer
3. The Network Load Balancer forwarded the request to the <ZillaPlus/> proxy
4. The <ZillaPlus/> proxy verified the client identity of the Kafka client
5. The <ZillaPlus/> proxy selected a matching client certificate to propagate client identity
6. The <ZillaPlus/> proxy routed the request to the appropriate MSK broker
7. The topic was created in the MSK broker
8. Public access was verified, authorized by trusted client certificate
1. This request was directed to the internet-facing Network Load Balancer
1. The Network Load Balancer forwarded the request to the <ZillaPlus/> proxy
1. The <ZillaPlus/> proxy verified the client identity of the Kafka client
1. The <ZillaPlus/> proxy selected a matching client certificate to propagate client identity
1. The <ZillaPlus/> proxy routed the request to the appropriate MSK broker
1. The topic was created in the MSK broker
1. Public access was verified, authorized by trusted client certificate

:::

<!-- @include: @partials/secure-public-access/send-message.md -->

## Conclusion

You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-sj4kquyndubiu) Secure Public Access. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../../aws-services/manage-cloudformation-stack.md) section.
You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) Secure Public Access. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../../aws-services/manage-cloudformation-stack.md) section.
Original file line number Diff line number Diff line change
Expand Up @@ -435,16 +435,16 @@ Replace these TLS bootstrap server names accordingly for your own custom wildcar
::: tip A quick summary of what just happened

1. The Kafka client with access to the public internet issued a request to create a new topic
2. This request was directed to the internet-facing Network Load Balancer
3. The Network Load Balancer forwarded the request to the <ZillaPlus/> proxy
4. The <ZillaPlus/> proxy routed the request to the appropriate MSK broker
5. The topic was created in the MSK broker
6. Public access was verified
1. This request was directed to the internet-facing Network Load Balancer
1. The Network Load Balancer forwarded the request to the <ZillaPlus/> proxy
1. The <ZillaPlus/> proxy routed the request to the appropriate MSK broker
1. The topic was created in the MSK broker
1. Public access was verified

:::

<!-- @include: @partials/secure-public-access/send-message.md -->

## Conclusion

You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-sj4kquyndubiu) Secure Public Access. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../../aws-services/manage-cloudformation-stack.md) section.
You have successfully deployed the [Zilla Plus for Amazon MSK](https://aws.amazon.com/marketplace/pp/prodview-jshnzslazfm44) Secure Public Access. Instructions on how to Monitor and Upgrade your <ZillaPlus/> proxy can be found in the [managing a cloudformation stack](../../aws-services/manage-cloudformation-stack.md) section.
2 changes: 1 addition & 1 deletion src/solutions/how-tos/aws-services/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
description: Common errors and how to handle them
---

# AWS Troubleshooting
# Troubleshooting

::: note Table of contents

Expand Down
Loading
Loading