MQTT docs updates and fixes

- clean up outputs - update link recommendation - fix double spaces - update kafka proxy conent and sidebar - update the mqtt intro configs - update route exit yaml example - point to new quickstart folder (aklivity#94) - Remove capabilities (aklivity#103) - Add server to options (aklivity#102) - update the new feature tag - boostrap fixes - add mqtt kafka proxy - add mqtt instructions for the quickstart
vordimous · Sep 21, 2023 · b16925d · b16925d
1 parent 71a5d0b
commit b16925d
Show file tree

Hide file tree

Showing 50 changed files with 821 additions and 286 deletions.
diff --git a/.config/vale/styles/Vocab/Base/accept.txt b/.config/vale/styles/Vocab/Base/accept.txt
@@ -1,17 +1,18 @@
+(?i)config
+(?i)datacenter
 (?i)http
 (?i)https
 (?i)jwt
 (?i)keystore
+(?i)middleware
+(?i)otlp
 (?i)pkcs12
 (?i)prometheus
+(?i)proxying
+(?i)realtime
 (?i)subnet(s)?
 (?i)todo
 (?i)truststore
-(?i)proxying
-(?i)realtime
-(?i)datacenter
-(?i)middleware
-(?i)config
 (?i)yaml
 acks
 Aiven
@@ -23,13 +24,15 @@ artifacthub
 Bidi
 bitnami
 BOMs
+CA(s)?
 cleartext
 CLIs
 config
 confluentinc
 CQRS
 cspell
 datagram
+declaratively
 docsearch
 etag
 fanout
@@ -38,6 +41,8 @@ gitea
 grpc
 grpcurl
 GSSAPI
+heroImage
+heroText
 hostname
 http
 idempotency
@@ -88,6 +93,3 @@ yaml
 Zilla
 zpm
 zpmw
-heroImage
-heroText
-declaratively
diff --git a/.config/vale/styles/Vocab/Docs/accept.txt b/.config/vale/styles/Vocab/Docs/accept.txt
diff --git a/.github/contributing/writing-guide.md b/.github/contributing/writing-guide.md
@@ -19,7 +19,7 @@ The docs will be organized according to the [Diataxis](https://diataxis.fr/) fra
 
 The Diataxis framework will dictate what content should be created and and keep the conent scoped to a purpose: Tutorials, How-Tos, Concepts, Reference.
 
-The user navigation can be a collection of any content organized by primary feature. This way a user can find the solution to their problem and  see different kinds of content all related to that solution.
+The user navigation can be a collection of any content organized by primary feature. This way a user can find the solution to their problem and see different kinds of content all related to that solution.
 
 ### File structure
 
@@ -62,6 +62,27 @@ Your users need reference material because they need truth and certainty - firm
 - **Get Started**: This is where users will start and learn what they need to be successful
 - **Reference**: This is an echo of the Diataxis definition and should remain as dry and generated as possible. The structure is set up for direct linking to individual components that readers may need more context on. Each component should have some sort of example to give context for it's usage
 
+### Links and References
+
+Links to other md files should use the local files path reference to the new document in order for any file render to correctly add links.
+
+ex: `[grpc-kafka](../../reference/config/bindings/binding-grpc-kafka.md#produce-capability)`
+
+[Reference-style links](https://www.markdownguide.org/basic-syntax/#reference-style-links) should only be used to clarify reading the raw text document when it is needed. This should be used in lists, tables, or complicated paragraphs. When used the reference definition should be placed in a group with other references and as close the the usage as makes sense. Exact highlighted context should be used unless there is a case when reuse of the same link is needed in the same section.
+
+Good examples improve readability without adding extra work for a plain text reader:
+
+- [vuejs-example]
+- [github-example]
+
+A bad example has no consistency which doesn't improve readability while navigation and maintenance are worse:
+
+- [electron-example]
+
+[vuejs-example]: https://github.com/vuejs/vue/blob/main/README.md?plain=1
+[github-example]: https://github.com/github/gitignore/blob/main/README.md?plain=1
+[electron-example]: https://github.com/electron/electron/blob/main/docs/faq.md?plain=1
+
 ## Writing & Grammar
 
 ### Style

diff --git a/src/.vuepress/sidebar/en.ts b/src/.vuepress/sidebar/en.ts
@@ -119,11 +119,11 @@ export const enSidebar = sidebar({
       ],
     },
     {
-      text: "Apache Kafka Proxying",
+      text: "Kafka Proxying",
       link: "concepts/kafka-proxies/rest-proxy.md",
       children: [
         {
-          text: "REST-Kafka Proxy",
+          text: "REST Kafka Proxy",
           collapsible: true,
           link: "concepts/kafka-proxies/rest-proxy.md",
           children: [
@@ -152,7 +152,7 @@ export const enSidebar = sidebar({
           ],
         },
         {
-          text: "SSE-Kafka Proxy",
+          text: "SSE Kafka Proxy",
           collapsible: true,
           link: "concepts/kafka-proxies/sse-proxy.md",
           children: [
@@ -181,7 +181,7 @@ export const enSidebar = sidebar({
           ],
         },
         {
-          text: "gRPC-Kafka Proxy",
+          text: "gRPC Kafka Proxy",
           collapsible: true,
           link: "concepts/kafka-proxies/grpc-proxy.md",
           children: [
@@ -195,6 +195,21 @@ export const enSidebar = sidebar({
             },
           ],
         },
+        {
+          text: "MQTT Kafka Proxy",
+          collapsible: true,
+          link: "concepts/kafka-proxies/mqtt-proxy.md",
+          children: [
+            {
+              text: "Overview",
+              link: "concepts/kafka-proxies/mqtt-proxy.md",
+            },
+            {
+              text: "Create a Simple MQTT Broker",
+              link: "tutorials/mqtt/mqtt-intro.md",
+            }
+          ],
+        },
         {
           text: "Amazon MSK Pubic Proxy",
           collapsible: true,

diff --git a/src/.vuepress/styles/index.scss b/src/.vuepress/styles/index.scss
@@ -104,3 +104,25 @@ table.no-head {
     }
 
 }
+
+
+.language-output {
+    border-radius: 0px !important;
+    background: transparent !important;
+
+    pre {
+        border-inline-start: 0.2rem solid #333;
+        margin: 1rem 0;
+        padding: 0.25rem 0 0.25rem 1rem;
+        font-size: 1rem;
+        overflow-wrap: break-word;
+        code {
+            color: #666 !important;
+
+        }
+    }
+
+    .copy-code-button {
+        display: none;
+    }
+}
diff --git a/src/concepts/kafka-proxies/grpc-proxy.md b/src/concepts/kafka-proxies/grpc-proxy.md
@@ -2,7 +2,7 @@
 description: This guide will walk through each unique gRPC message request and response design and how Zilla is configured to manage the connection for each.
 ---
 
-# gRPC Proxy
+# gRPC Kafka Proxy
 
 This guide will walk through each unique gRPC message request and response design and how Zilla is configured to manage the connection for each.
 

diff --git a/src/concepts/kafka-proxies/mqtt-proxy.md b/src/concepts/kafka-proxies/mqtt-proxy.md
@@ -0,0 +1,114 @@
+---
+description: This guide will walk through the way Zilla manages MQTT Pub/Sub connections and messages.
+---
+
+# MQTT Kafka Proxy
+
+This guide will walk through the way Zilla manages MQTT Pub/Sub connections and messages.
+
+An MQTT server acts as a broker between publishers and subscribers. This requires a complex protocol to manage the wide range of IoT devices and use cases. By proxying these messages on and off of Kafka with the [mqtt-kafka](../../reference/config/bindings/binding-mqtt-kafka.md) binding, IoT devices can transmit data to a wider range of tech stacks, adapting to more business needs.
+
+Unlike other proxies, Zilla manages the different MQTT topics instead of blindly passing them down to Kafka. This way the Kafka architecture can be optimized for MQTT Pub/Sub. MQTT client subscribers and publishers will communicate with Zilla the same as any broker.
+
+## Step 1: Declaring the broker
+
+A Zilla MQTT server can manage client sessions and broker all of the messages sent.
+
+```yaml
+mqtt_server:
+  type: mqtt
+  kind: server
+  exit: mqtt_kafka_proxy
+
+mqtt_kafka_proxy:
+  type: mqtt-kafka
+  kind: proxy
+  options:
+    topics:
+      sessions: mqtt-sessions
+      messages: mqtt-messages
+      retained: mqtt-retained
+```
+
+### Protocol version
+
+The Zilla MQTT `server` supports the [MQTT v5.0 Specification].
+
+::: info Feature Coming Soon <HopeIcon icon="circle-right"/>
+[MQTT v3.1.1 Specification] support is currently on the [Zilla roadmap]. Star and watch the [Zilla repo] for new releases!
+:::
+
+[MQTT v5.0 Specification]:https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html
+[MQTT v3.1.1 Specification]:http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html
+[Zilla roadmap]:https://github.com/orgs/aklivity/projects/4
+[Zilla repo]:https://github.com/aklivity/zilla/releases
+
+### QOS
+
+The Zilla MQTT `server` supports the "At most once (QoS 0)" Quality of Service flag.
+
+::: info Feature Coming Soon <HopeIcon icon="circle-right"/>
+At least once (QoS 1) and Exactly once (QoS 2) delivery will be support is currently on the [Zilla roadmap]. Star and watch the [Zilla repo] for new releases!
+:::
+
+## Step 2: Pub/Sub message reflect with Kafka
+
+Zilla manages MQTT pub/sub using three kafka topics. The specific topic names can be configured using the [options.topics](../../reference/config/bindings/binding-mqtt-kafka.md#options-topics) property.
+
+```yaml
+topics:
+  messages: mqtt-messages
+  retained: mqtt-retained
+  sessions: mqtt-sessions
+```
+
+### Messages on Kafka
+
+All MQTT messages brokered by Zilla are published on the `messages` Kafka topic. The MQTT message topic becomes the kafka key
+
+### Retaining Messages
+
+MQTT messages with the `retain` flag set will have a copy published on the `retained` Kafka topic.
+
+### Session Management
+
+MQTT connect and disconnect messages are published on the `sessions` Kafka topic.
+
+## Step 3: Authorizing clients
+
+A client connection to the MQTT server can be guarded by the [jwt](../../reference/config/guards/guard-jwt.md) guard.
+
+```yaml{2,19,25}
+guards:
+  jwt_mqtt_auth:
+    type: jwt
+    options:
+      issuer: https://auth.example.com
+      audience: https://api.example.com
+      keys:
+        - kty: RSA
+          n: qq...aDQ==
+          e: AQAB
+          alg: RS256
+          kid: example
+bindings:
+  mqtt_server:
+    type: mqtt
+    kind: server
+    options:
+      authorization:
+        jwt_mqtt_auth:
+          credentials:
+            connect:
+              username: Bearer {credentials}
+    routes:
+      - guarded:
+          jwt_mqtt_auth:
+            - mqtt:stream
+        exit: mqtt_kafka_proxy
+
+```
+
+## Try it out
+
+Go check out the [mqtt.kafka.reflect](https://github.com/aklivity/zilla-examples/tree/main/mqtt.kafka.reflect) example for a full implementation of an EchoService.
diff --git a/src/concepts/kafka-proxies/rest-proxy.md b/src/concepts/kafka-proxies/rest-proxy.md
@@ -1,4 +1,8 @@
-# REST Proxy
+---
+description: Zilla lets you configure application-centric REST API endpoints that unlock Kafka event-driven architectures.
+---
+
+# REST Kafka Proxy
 <!-- TODO enable -->
 <!-- markdownlint-disable -->
 
@@ -104,7 +108,7 @@ bindings:
 
 ### CORS
 
-Zilla supports Cross-Origin Resource Sharing (CORS)  and allows you to specify fine-grained access control including specific request origins, methods and headers allowed, and specific response headers exposed. Since it acts more like a guard and has no dependency on Apache Kafka configuration, you need to define it in the [http binding](../../reference/config/bindings/binding-http.md)
+Zilla supports Cross-Origin Resource Sharing (CORS) and allows you to specify fine-grained access control including specific request origins, methods and headers allowed, and specific response headers exposed. Since it acts more like a guard and has no dependency on Apache Kafka configuration, you need to define it in the [http binding](../../reference/config/bindings/binding-http.md)
 
 ### zilla.yaml
 
@@ -132,7 +136,7 @@ http_server:
 
 ### Authorization
 
-Since `Zilla` config is very much modular it has the concept of [`guard`](../../reference/config/overview.md#guards) where you define your `guard` configuration and reference that `guard` to authorize a specific endpoint. Currently, `Zilla`  supports [`JSON Web Token (JWT)`](../../reference/config/guards/guard-jwt.md)  mechanism to authorize the endpoint.
+Since `Zilla` config is very much modular it has the concept of [`guard`](../../reference/config/overview.md#guards) where you define your `guard` configuration and reference that `guard` to authorize a specific endpoint. Currently, `Zilla`  supports [`JSON Web Token (JWT)`](../../reference/config/guards/guard-jwt.md) mechanism to authorize the endpoint.
 
 The information about keys and other details such as issuer and audience you can get from `JWT` providers for example in the case of Auth0 you can use the command below.