From f356e1fb6c14fb66516ad902172a395dd371ae76 Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 18 Nov 2023 13:19:23 +0530 Subject: [PATCH 01/43] initial structure --- .../kafka/managing-schemas-using-schema-registry.md | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md new file mode 100644 index 00000000000..f8898ceeee8 --- /dev/null +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -0,0 +1,11 @@ +## Introduction + +## Background context + +## Prerequisites + +## Steps + +## Summary + +## Next steps From 7db40f4b5a6d022b0ab643dc9865546c132936e8 Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 18 Nov 2023 13:34:42 +0530 Subject: [PATCH 02/43] added metadata --- .../kafka/managing-schemas-using-schema-registry.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index f8898ceeee8..d35b7de4f88 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -1,3 +1,9 @@ +--- +title: Managing schemas using Schema Registry +description: The guide explains how you can manage schemas using Schema Registry. +weight: 30 +--- + ## Introduction ## Background context From 90e32623b49d221b7bef11ac53c5f629d979ffe9 Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 18 Nov 2023 13:35:41 +0530 Subject: [PATCH 03/43] added introduction --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 1 + 1 file changed, 1 insertion(+) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index d35b7de4f88..0d04d80edeb 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -5,6 +5,7 @@ weight: 30 --- ## Introduction +This tutorial will teach you how to manage schemas with Schema Registry. This will enable us to ensure a consistent exchange of data across distributed systems, thus minimizing any errors. ## Background context From c0f04c7b52b8ebfbc40bf96a9ec991bcaaee4a3e Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 18 Nov 2023 13:37:25 +0530 Subject: [PATCH 04/43] changed page weight --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 0d04d80edeb..991af1a8f2e 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -1,7 +1,7 @@ --- title: Managing schemas using Schema Registry description: The guide explains how you can manage schemas using Schema Registry. -weight: 30 +weight: 50 --- ## Introduction From dd81c6d11fd334e9d0ae9076dbdb88a29d085cf6 Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 18 Nov 2023 13:55:44 +0530 Subject: [PATCH 05/43] added background context --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 991af1a8f2e..76959409685 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -8,6 +8,9 @@ weight: 50 This tutorial will teach you how to manage schemas with Schema Registry. This will enable us to ensure a consistent exchange of data across distributed systems, thus minimizing any errors. ## Background context +To handle the evolving complexity of modern Event-Driven Architecture, the need for schema management has become increasingly prevalent. Data validation is a technique that helps in maintaining data consistency and compatibility. Schema Registry can be used to validate data being exchanged by producers and consumers. + +In this tutorial, we’ve used the Apicurio Registry, which is a popular Schema Registry implementation, in combination with Avro which is a language-neutral data serialization system. ## Prerequisites From c7eac29647209aefbfcaf2a873eb2e215c849e17 Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 18 Nov 2023 17:04:39 +0530 Subject: [PATCH 06/43] added prerequisites --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 76959409685..55f42d858a7 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -13,6 +13,10 @@ To handle the evolving complexity of modern Event-Driven Architecture, the need In this tutorial, we’ve used the Apicurio Registry, which is a popular Schema Registry implementation, in combination with Avro which is a language-neutral data serialization system. ## Prerequisites +Before you begin with this tutorial, it is already assumed that you already have: +- Set up an instance of Apicurio Registry +- Created an AsyncAPI document +- Have some knowledge of Avro serialization ## Steps From 5f4744cae82a76c76f9cc84b38a0aa3dd1cceea4 Mon Sep 17 00:00:00 2001 From: unknown Date: Sun, 19 Nov 2023 14:47:57 +0530 Subject: [PATCH 07/43] added temp files for preview --- pages/docs/tutorials/kafka/_section.md | 4 ++++ pages/docs/tutorials/kafka/index.md | 4 ++++ .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 3 files changed, 9 insertions(+), 1 deletion(-) create mode 100644 pages/docs/tutorials/kafka/_section.md create mode 100644 pages/docs/tutorials/kafka/index.md diff --git a/pages/docs/tutorials/kafka/_section.md b/pages/docs/tutorials/kafka/_section.md new file mode 100644 index 00000000000..994e7b8144a --- /dev/null +++ b/pages/docs/tutorials/kafka/_section.md @@ -0,0 +1,4 @@ +--- +title: 'Kafka' +weight: 2 +--- \ No newline at end of file diff --git a/pages/docs/tutorials/kafka/index.md b/pages/docs/tutorials/kafka/index.md new file mode 100644 index 00000000000..e204dbc49ce --- /dev/null +++ b/pages/docs/tutorials/kafka/index.md @@ -0,0 +1,4 @@ +--- +title: Tutorial-1 +weight: 20 +--- \ No newline at end of file diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 55f42d858a7..4a0e497180e 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -16,7 +16,7 @@ In this tutorial, we’ve used the Apicurio Registry, which is a popular Schema Before you begin with this tutorial, it is already assumed that you already have: - Set up an instance of Apicurio Registry - Created an AsyncAPI document -- Have some knowledge of Avro serialization +- Some knowledge of Avro serialization ## Steps From e0a693caefea87285f9ca1073d4c7d8b519ab21c Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Mon, 27 Nov 2023 16:35:03 +0530 Subject: [PATCH 08/43] Update pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Fran Méndez --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 4a0e497180e..0a0aefe8034 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -1,6 +1,6 @@ --- title: Managing schemas using Schema Registry -description: The guide explains how you can manage schemas using Schema Registry. +description: This guide explains how you can manage schemas using Schema Registry. weight: 50 --- From 07f8df569065d6730e78f612d089e834bbd5f08d Mon Sep 17 00:00:00 2001 From: unknown Date: Wed, 6 Dec 2023 20:44:05 +0530 Subject: [PATCH 09/43] implemented feedback --- .../kafka/managing-schemas-using-schema-registry.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 0a0aefe8034..292ba982f33 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -8,18 +8,15 @@ weight: 50 This tutorial will teach you how to manage schemas with Schema Registry. This will enable us to ensure a consistent exchange of data across distributed systems, thus minimizing any errors. ## Background context -To handle the evolving complexity of modern Event-Driven Architecture, the need for schema management has become increasingly prevalent. Data validation is a technique that helps in maintaining data consistency and compatibility. Schema Registry can be used to validate data being exchanged by producers and consumers. +The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains the schemas for data exchanged between various components of a modern distributed system. Validating data being exchanged helps in maintaining data consistency and compatibility. -In this tutorial, we’ve used the Apicurio Registry, which is a popular Schema Registry implementation, in combination with Avro which is a language-neutral data serialization system. +While several Schema Registry implementations exist, we will be using the Apicurio Registry in this tutorial. Apicurio Registry is a popular open source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. We will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. -## Prerequisites -Before you begin with this tutorial, it is already assumed that you already have: -- Set up an instance of Apicurio Registry -- Created an AsyncAPI document -- Some knowledge of Avro serialization +# Installation guide ## Steps ## Summary ## Next steps +In the next tutorial, we are going to learn about Kafka bindings. \ No newline at end of file From 02a622b2aa83fa1152fc465c8f6088ae915f588d Mon Sep 17 00:00:00 2001 From: unknown Date: Wed, 6 Dec 2023 20:53:45 +0530 Subject: [PATCH 10/43] added steps --- .../managing-schemas-using-schema-registry.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 292ba982f33..3b94d204921 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -5,16 +5,26 @@ weight: 50 --- ## Introduction -This tutorial will teach you how to manage schemas with Schema Registry. This will enable us to ensure a consistent exchange of data across distributed systems, thus minimizing any errors. +This tutorial will teach you how to manage schemas with Schema Registry. This will enable us to ensure a consistent data exchange across distributed systems, thus minimizing errors. ## Background context The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains the schemas for data exchanged between various components of a modern distributed system. Validating data being exchanged helps in maintaining data consistency and compatibility. -While several Schema Registry implementations exist, we will be using the Apicurio Registry in this tutorial. Apicurio Registry is a popular open source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. We will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. +While several Schema Registry implementations exist, we will use the Apicurio Registry in this tutorial. Apicurio Registry is a popular open source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. We will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. # Installation guide ## Steps +1. Install [Docker](https://docs.docker.com/engine/install/) from the official website. +2. Open the command prompt and pull up the Apicurio Registry Docker Image from Docker Hub using the following command: + + {`docker pull apicurio/apicurio-registry`} + +3. Run Apicurio Registry Docker Image in a Docker container by using the command: + + {`docker run -it -p 8080:8080 apicurio/apicurio-registry:latest-snapshot`} + +4. Access Apicurio Registry once the container is running by navigating to http://localhost:8080. ## Summary From b3a74d44c345f7e1eb26530bd79046ea128ab66c Mon Sep 17 00:00:00 2001 From: unknown Date: Wed, 6 Dec 2023 21:33:44 +0530 Subject: [PATCH 11/43] added installation steps --- .../managing-schemas-using-schema-registry.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 3b94d204921..39542a195f7 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -1,30 +1,30 @@ --- title: Managing schemas using Schema Registry -description: This guide explains how you can manage schemas using Schema Registry. +description: The guide explains how you can manage schemas using Schema Registry. weight: 50 --- ## Introduction -This tutorial will teach you how to manage schemas with Schema Registry. This will enable us to ensure a consistent data exchange across distributed systems, thus minimizing errors. +You will learn how to manage schemas with Schema Registry. Managing schemas will enable us to ensure a consistent data exchange across distributed systems, thus minimizing errors. ## Background context The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains the schemas for data exchanged between various components of a modern distributed system. Validating data being exchanged helps in maintaining data consistency and compatibility. While several Schema Registry implementations exist, we will use the Apicurio Registry in this tutorial. Apicurio Registry is a popular open source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. We will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. -# Installation guide - -## Steps +## Installation guide 1. Install [Docker](https://docs.docker.com/engine/install/) from the official website. 2. Open the command prompt and pull up the Apicurio Registry Docker Image from Docker Hub using the following command: {`docker pull apicurio/apicurio-registry`} -3. Run Apicurio Registry Docker Image in a Docker container by using the command: + +## Steps +1. Run Apicurio Registry Docker Image in a Docker container by using the command: {`docker run -it -p 8080:8080 apicurio/apicurio-registry:latest-snapshot`} -4. Access Apicurio Registry once the container is running by navigating to http://localhost:8080. +2. Access Apicurio Registry once the container is running by navigating to http://localhost:8080. ## Summary From fe4f735d8a7c9bfc77b3f8a81802a9aa0b6fd7fc Mon Sep 17 00:00:00 2001 From: unknown Date: Wed, 6 Dec 2023 21:36:18 +0530 Subject: [PATCH 12/43] removed mentions of we according to the style guide --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 39542a195f7..6b90b4ec371 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -10,7 +10,7 @@ You will learn how to manage schemas with Schema Registry. Managing schemas will ## Background context The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains the schemas for data exchanged between various components of a modern distributed system. Validating data being exchanged helps in maintaining data consistency and compatibility. -While several Schema Registry implementations exist, we will use the Apicurio Registry in this tutorial. Apicurio Registry is a popular open source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. We will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. +While several Schema Registry implementations exist, you will use the Apicurio Registry in this tutorial. Apicurio Registry is a popular open-source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. You will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. ## Installation guide 1. Install [Docker](https://docs.docker.com/engine/install/) from the official website. @@ -29,4 +29,4 @@ While several Schema Registry implementations exist, we will use the Apicurio Re ## Summary ## Next steps -In the next tutorial, we are going to learn about Kafka bindings. \ No newline at end of file +In the next tutorial, you are going to learn about Kafka bindings. \ No newline at end of file From 4aa9887b4bc6ee45f43bee6ef6309a13ca81cf8b Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Thu, 21 Dec 2023 22:43:58 +0530 Subject: [PATCH 13/43] Update managing-schemas-using-schema-registry.md --- .../kafka/managing-schemas-using-schema-registry.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 6b90b4ec371..15f4f944836 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -14,10 +14,6 @@ While several Schema Registry implementations exist, you will use the Apicurio R ## Installation guide 1. Install [Docker](https://docs.docker.com/engine/install/) from the official website. -2. Open the command prompt and pull up the Apicurio Registry Docker Image from Docker Hub using the following command: - - {`docker pull apicurio/apicurio-registry`} - ## Steps 1. Run Apicurio Registry Docker Image in a Docker container by using the command: @@ -29,4 +25,4 @@ While several Schema Registry implementations exist, you will use the Apicurio R ## Summary ## Next steps -In the next tutorial, you are going to learn about Kafka bindings. \ No newline at end of file +In the next tutorial, you are going to learn about Kafka bindings. From 398e6bd6b0218b6375409ff29c63e1be1f3245fa Mon Sep 17 00:00:00 2001 From: unknown Date: Thu, 11 Jan 2024 17:58:08 +0530 Subject: [PATCH 14/43] added curl command for creating avro schema artifact --- .../managing-schemas-using-schema-registry.md | 27 ++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 15f4f944836..2411cd844bf 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -20,7 +20,32 @@ While several Schema Registry implementations exist, you will use the Apicurio R {`docker run -it -p 8080:8080 apicurio/apicurio-registry:latest-snapshot`} -2. Access Apicurio Registry once the container is running by navigating to http://localhost:8080. +2. Once your Apicurio Registry is running, you can upload the schema in another terminal window. Create an Avro Schema Artifact using the following command by replacing {Avro_Schema} with your Avro Schema: + +curl \ +http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ +-X POST \ +-H "Content-Type: application/json; artifactType=AVRO" \ +-H "X-Registry-ArtifactId: UserSignedUp" \ +--data @- << EOF +{ + "type": "record", + "name": "UserSignedUp", + "namespace": "com.company", + "doc": "User sign-up information", + "fields": [ + { + "name": "userId", + "type": "int" + }, + { + "name": "userEmail", + "type": "string" + } + ] +} +EOF + ## Summary From f5f397084d839932688ea2a8ea417c5131de3ab4 Mon Sep 17 00:00:00 2001 From: unknown Date: Thu, 11 Jan 2024 21:54:00 +0530 Subject: [PATCH 15/43] added usage of --- .../kafka/managing-schemas-using-schema-registry.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 2411cd844bf..496bea78f4b 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -20,7 +20,7 @@ While several Schema Registry implementations exist, you will use the Apicurio R {`docker run -it -p 8080:8080 apicurio/apicurio-registry:latest-snapshot`} -2. Once your Apicurio Registry is running, you can upload the schema in another terminal window. Create an Avro Schema Artifact using the following command by replacing {Avro_Schema} with your Avro Schema: +2. Once your Apicurio Registry is running, you can upload the schema in another terminal window. Create an Avro Schema Artifact using the following command: curl \ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ @@ -46,6 +46,11 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ } EOF +3. You can download the Avro schema by visiting the URL below and replacing {UserSignedUp} with the id of the schema used in the previous step. +http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/{UserSignedUp} +4. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: +$ref: {url} +Replace {url} with the url from the previous step. ## Summary From 719c97a890d9c8c8f2067cc45cb0dba196887445 Mon Sep 17 00:00:00 2001 From: unknown Date: Thu, 11 Jan 2024 21:54:00 +0530 Subject: [PATCH 16/43] added usage of json pointer --- .../kafka/managing-schemas-using-schema-registry.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 2411cd844bf..496bea78f4b 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -20,7 +20,7 @@ While several Schema Registry implementations exist, you will use the Apicurio R {`docker run -it -p 8080:8080 apicurio/apicurio-registry:latest-snapshot`} -2. Once your Apicurio Registry is running, you can upload the schema in another terminal window. Create an Avro Schema Artifact using the following command by replacing {Avro_Schema} with your Avro Schema: +2. Once your Apicurio Registry is running, you can upload the schema in another terminal window. Create an Avro Schema Artifact using the following command: curl \ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ @@ -46,6 +46,11 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ } EOF +3. You can download the Avro schema by visiting the URL below and replacing {UserSignedUp} with the id of the schema used in the previous step. +http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/{UserSignedUp} +4. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: +$ref: {url} +Replace {url} with the url from the previous step. ## Summary From 83b2f33f73acda0f90e80aa3c9eb076fd888b41a Mon Sep 17 00:00:00 2001 From: unknown Date: Thu, 11 Jan 2024 22:14:30 +0530 Subject: [PATCH 17/43] specified apicurio registry version in command --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 496bea78f4b..c0e5c2e6e7c 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -18,7 +18,7 @@ While several Schema Registry implementations exist, you will use the Apicurio R ## Steps 1. Run Apicurio Registry Docker Image in a Docker container by using the command: - {`docker run -it -p 8080:8080 apicurio/apicurio-registry:latest-snapshot`} + {`docker run -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.7.Final`} 2. Once your Apicurio Registry is running, you can upload the schema in another terminal window. Create an Avro Schema Artifact using the following command: From 9da1a665d61f2899d380eae0d12db353c5fdf743 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Tue, 16 Jan 2024 15:04:29 +0530 Subject: [PATCH 18/43] Update pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md Co-authored-by: Lukasz Gornicki --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index c0e5c2e6e7c..288ae842783 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -20,7 +20,7 @@ While several Schema Registry implementations exist, you will use the Apicurio R {`docker run -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.7.Final`} -2. Once your Apicurio Registry is running, you can upload the schema in another terminal window. Create an Avro Schema Artifact using the following command: +2. Once your local instance of Apicurio Registry is running, you can upload your Avro schema to it. Open new terminal window and create an Avro schema artifact using the following command: curl \ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ From 0230f93c9d72e2eed1a7eea4c012b53ba05fe8a0 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Tue, 16 Jan 2024 15:05:14 +0530 Subject: [PATCH 19/43] Update pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md Co-authored-by: Lukasz Gornicki --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 288ae842783..529b4210523 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -16,7 +16,7 @@ While several Schema Registry implementations exist, you will use the Apicurio R 1. Install [Docker](https://docs.docker.com/engine/install/) from the official website. ## Steps -1. Run Apicurio Registry Docker Image in a Docker container by using the command: +1. Start Apicurio Registry locally using below docker command: {`docker run -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.7.Final`} From 8794a718a4e3e594f5da8c8d7c56ec8839714f11 Mon Sep 17 00:00:00 2001 From: unknown Date: Tue, 16 Jan 2024 15:08:58 +0530 Subject: [PATCH 20/43] added link to apicurio registry official docs --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 529b4210523..c4b518c9afe 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -10,7 +10,7 @@ You will learn how to manage schemas with Schema Registry. Managing schemas will ## Background context The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains the schemas for data exchanged between various components of a modern distributed system. Validating data being exchanged helps in maintaining data consistency and compatibility. -While several Schema Registry implementations exist, you will use the Apicurio Registry in this tutorial. Apicurio Registry is a popular open-source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. You will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. +While several Schema Registry implementations exist, you will use the [Apicurio Registry](https://www.apicur.io/registry/) in this tutorial. Apicurio Registry is a popular open-source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. You will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. ## Installation guide 1. Install [Docker](https://docs.docker.com/engine/install/) from the official website. From 54b67f023863d9ea7c2ab7eb5da52a28e0d2c084 Mon Sep 17 00:00:00 2001 From: unknown Date: Thu, 18 Jan 2024 21:11:32 +0530 Subject: [PATCH 21/43] updated docker command --- .../kafka/managing-schemas-using-schema-registry.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index c4b518c9afe..fef9e034f8e 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -18,7 +18,7 @@ While several Schema Registry implementations exist, you will use the [Apicurio ## Steps 1. Start Apicurio Registry locally using below docker command: - {`docker run -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.7.Final`} + {`docker run --env CORS_ALLOWED_ORIGINS=* -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final`} 2. Once your local instance of Apicurio Registry is running, you can upload your Avro schema to it. Open new terminal window and create an Avro schema artifact using the following command: @@ -46,9 +46,9 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ } EOF -3. You can download the Avro schema by visiting the URL below and replacing {UserSignedUp} with the id of the schema used in the previous step. +1. You can download the Avro schema by visiting the URL below and replacing {UserSignedUp} with the id of the schema used in the previous step. http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/{UserSignedUp} -4. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: +1. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: $ref: {url} Replace {url} with the url from the previous step. From 0aa96eaacc8a05457563b3af3da220e61e6188f3 Mon Sep 17 00:00:00 2001 From: unknown Date: Sun, 21 Jan 2024 21:26:52 +0530 Subject: [PATCH 22/43] added url for downloading avro schema --- .../kafka/managing-schemas-using-schema-registry.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index fef9e034f8e..f827fc5c4d0 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -46,9 +46,9 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ } EOF -1. You can download the Avro schema by visiting the URL below and replacing {UserSignedUp} with the id of the schema used in the previous step. -http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/{UserSignedUp} -1. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: +3. You can download the Avro schema by visiting the URL below: +http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp +4. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: $ref: {url} Replace {url} with the url from the previous step. From ab52ef6ae437055ffa001f4dcd51dea3b7d8f194 Mon Sep 17 00:00:00 2001 From: unknown Date: Tue, 23 Jan 2024 13:55:23 +0530 Subject: [PATCH 23/43] added prerequisites section --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index f827fc5c4d0..987a2aca136 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -12,8 +12,8 @@ The need for schema management has become increasingly prevalent to handle the e While several Schema Registry implementations exist, you will use the [Apicurio Registry](https://www.apicur.io/registry/) in this tutorial. Apicurio Registry is a popular open-source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. You will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. -## Installation guide -1. Install [Docker](https://docs.docker.com/engine/install/) from the official website. +## Prerequisites +Install [Docker](https://docs.docker.com/engine/install/) from the official website. ## Steps 1. Start Apicurio Registry locally using below docker command: From 40c64841bf2b8ac3c2a85a276d12ac7a679d31b7 Mon Sep 17 00:00:00 2001 From: unknown Date: Tue, 23 Jan 2024 13:57:59 +0530 Subject: [PATCH 24/43] changed optional step to remember block --- .../kafka/managing-schemas-using-schema-registry.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 987a2aca136..4428505e7f2 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -46,9 +46,11 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ } EOF -3. You can download the Avro schema by visiting the URL below: -http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp -4. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: + +Download your Avro schema by visiting the following URL: +http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. + +3. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: $ref: {url} Replace {url} with the url from the previous step. From 55a59a357a721cb209e7ee02190b095d9ee0b174 Mon Sep 17 00:00:00 2001 From: unknown Date: Tue, 23 Jan 2024 14:09:14 +0530 Subject: [PATCH 25/43] added summary --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 1 + 1 file changed, 1 insertion(+) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 4428505e7f2..94a4dbe0d88 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -55,6 +55,7 @@ $ref: {url} Replace {url} with the url from the previous step. ## Summary +In this tutorial you’ve gained insights about managing Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. It is important to note that this approach is valid for various other schema types. ## Next steps In the next tutorial, you are going to learn about Kafka bindings. From f78f7c7e512c6d030bd975e4b2ab8d1dc8d3dcfb Mon Sep 17 00:00:00 2001 From: unknown Date: Fri, 26 Jan 2024 13:36:54 +0530 Subject: [PATCH 26/43] fixed link to previous tutorial --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 94a4dbe0d88..637ebe0dc00 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -50,7 +50,7 @@ EOF Download your Avro schema by visiting the following URL: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. -3. An alternative to having the schema in a separate file as in (Joy's tutorial), would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: +1. An alternative to having the schema in a separate file as you learned in the [previous tutorial](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: $ref: {url} Replace {url} with the url from the previous step. @@ -58,4 +58,4 @@ Replace {url} with the url from the previous step. In this tutorial you’ve gained insights about managing Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. It is important to note that this approach is valid for various other schema types. ## Next steps -In the next tutorial, you are going to learn about Kafka bindings. +In the next tutorial, you are going to learn about Kafka bindings. \ No newline at end of file From b8261159ad7fc94c67657c80e1fbf826515ded8f Mon Sep 17 00:00:00 2001 From: unknown Date: Fri, 26 Jan 2024 13:41:33 +0530 Subject: [PATCH 27/43] added code snippet --- .../managing-schemas-using-schema-registry.md | 49 ++++++++++++++++--- 1 file changed, 43 insertions(+), 6 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 637ebe0dc00..f370b1136b8 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -17,11 +17,14 @@ Install [Docker](https://docs.docker.com/engine/install/) from the official webs ## Steps 1. Start Apicurio Registry locally using below docker command: - - {`docker run --env CORS_ALLOWED_ORIGINS=* -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final`} - + +``` +{`docker run --env CORS_ALLOWED_ORIGINS=* -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final`} +``` + 2. Once your local instance of Apicurio Registry is running, you can upload your Avro schema to it. Open new terminal window and create an Avro schema artifact using the following command: - + +``` curl \ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ -X POST \ @@ -45,15 +48,49 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts \ ] } EOF - +``` + Download your Avro schema by visiting the following URL: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. -1. An alternative to having the schema in a separate file as you learned in the [previous tutorial](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: + +3. An alternative to having the schema in a separate file as you learned in the [previous tutorial](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: $ref: {url} Replace {url} with the url from the previous step. +``` +asyncapi: 3.0.0 +info: + title: User Signup API + version: 1.0.0 + description: The API notifies you whenever a new user signs up in the application. +servers: + kafkaServer: + host: test.mykafkacluster.org:8092 + description: Kafka Server + protocol: kafka +operations: + onUserSignedUp: + action: receive + channel: + $ref: '#/channels/userSignedUp' +channels: + userSignedUp: + description: This channel contains a message per each user who signs up in our application. + address: user_signedup + messages: + userSignedUp: + $ref: '#/components/messages/userSignedUp' +components: + messages: + userSignedUp: + payload: + schemaFormat: 'application/vnd.apache.avro+json;version=1.9.0' + schema: + $ref: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp +``` + ## Summary In this tutorial you’ve gained insights about managing Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. It is important to note that this approach is valid for various other schema types. From 41907b5f8c45ec90c3a5121f0889abd9e5af99ea Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 27 Jan 2024 12:21:46 +0530 Subject: [PATCH 28/43] added next steps --- .../kafka/managing-schemas-using-schema-registry.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index f370b1136b8..254ce7f13fe 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -95,4 +95,9 @@ components: In this tutorial you’ve gained insights about managing Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. It is important to note that this approach is valid for various other schema types. ## Next steps -In the next tutorial, you are going to learn about Kafka bindings. \ No newline at end of file +Now that you have learnt how to manage schemas, check out our tutorial for [bindings with Kafka](/docs/tutorials/kafka/bindings-with-kafka) to start sending messages between your services. + + + + + From aba245b1b1108a33f1acd086938b67ede8ae6220 Mon Sep 17 00:00:00 2001 From: unknown Date: Tue, 30 Jan 2024 23:20:10 +0530 Subject: [PATCH 29/43] changed description --- .../kafka/managing-schemas-using-schema-registry.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 254ce7f13fe..04688488d64 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -1,6 +1,6 @@ --- title: Managing schemas using Schema Registry -description: The guide explains how you can manage schemas using Schema Registry. +description: The tutorial explains how you can manage schemas using Schema Registry. weight: 50 --- @@ -95,9 +95,4 @@ components: In this tutorial you’ve gained insights about managing Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. It is important to note that this approach is valid for various other schema types. ## Next steps -Now that you have learnt how to manage schemas, check out our tutorial for [bindings with Kafka](/docs/tutorials/kafka/bindings-with-kafka) to start sending messages between your services. - - - - - +Now that you have learnt how to manage schemas, check out our tutorial for [bindings with Kafka](/docs/tutorials/kafka/bindings-with-kafka) to start sending messages between your services. \ No newline at end of file From fb0a1fafb4145e429927271cf31383c474583811 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Wed, 7 Feb 2024 13:18:06 +0530 Subject: [PATCH 30/43] Added heading to each step --- .../kafka/managing-schemas-using-schema-registry.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 04688488d64..ec4f6c22da6 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -16,12 +16,15 @@ While several Schema Registry implementations exist, you will use the [Apicurio Install [Docker](https://docs.docker.com/engine/install/) from the official website. ## Steps + +### Start Apicurio Registry 1. Start Apicurio Registry locally using below docker command: ``` {`docker run --env CORS_ALLOWED_ORIGINS=* -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final`} ``` +### Upload Avro Schema 2. Once your local instance of Apicurio Registry is running, you can upload your Avro schema to it. Open new terminal window and create an Avro schema artifact using the following command: ``` @@ -55,6 +58,7 @@ Download your Avro schema by visiting the following URL: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. +### Replace link using JSON pointer 3. An alternative to having the schema in a separate file as you learned in the [previous tutorial](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: $ref: {url} Replace {url} with the url from the previous step. @@ -95,4 +99,4 @@ components: In this tutorial you’ve gained insights about managing Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. It is important to note that this approach is valid for various other schema types. ## Next steps -Now that you have learnt how to manage schemas, check out our tutorial for [bindings with Kafka](/docs/tutorials/kafka/bindings-with-kafka) to start sending messages between your services. \ No newline at end of file +Now that you have learnt how to manage schemas, check out our tutorial for [bindings with Kafka](/docs/tutorials/kafka/bindings-with-kafka) to start sending messages between your services. From 58f11ef6e99b6ed9d4531630c74e86aed6cbe675 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Wed, 7 Feb 2024 13:24:51 +0530 Subject: [PATCH 31/43] fixed anchor text --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index ec4f6c22da6..50cf9334b41 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -59,7 +59,7 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. ### Replace link using JSON pointer -3. An alternative to having the schema in a separate file as you learned in the [previous tutorial](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: +3. An alternative to having the schema in a separate file as you learned in [Describe Kafka message payload using Avro Schema](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: $ref: {url} Replace {url} with the url from the previous step. From 704dab509174488979c1725439c897e3ec66afa0 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Wed, 7 Feb 2024 13:30:42 +0530 Subject: [PATCH 32/43] added url --- .../kafka/managing-schemas-using-schema-registry.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 50cf9334b41..ef32d994a65 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -59,9 +59,10 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. ### Replace link using JSON pointer -3. An alternative to having the schema in a separate file as you learned in [Describe Kafka message payload using Avro Schema](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and replace it with the following: -$ref: {url} -Replace {url} with the url from the previous step. +3. An alternative to having the schema in a separate file as you learned in [Describe Kafka message payload using Avro Schema](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and a $ref that points to the url from the previous step. +``` +$ref: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp +``` ``` asyncapi: 3.0.0 From d9c3a03ed2260f5594ad5a48d71be75a2085cca0 Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Thu, 8 Feb 2024 18:23:09 -0800 Subject: [PATCH 33/43] tw editorial review --- .../managing-schemas-using-schema-registry.md | 21 +++++++++---------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index ef32d994a65..7a1fef5576b 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -1,31 +1,30 @@ --- title: Managing schemas using Schema Registry -description: The tutorial explains how you can manage schemas using Schema Registry. +description: A tutorial teaching how to manage schemas using Schema Registry. weight: 50 --- ## Introduction -You will learn how to manage schemas with Schema Registry. Managing schemas will enable us to ensure a consistent data exchange across distributed systems, thus minimizing errors. +In this tutorial, you'll learn how to manage schemas with Schema Registry. Managing schemas ensures a consistent data exchange across distributed systems, thus minimizing errors. ## Background context -The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains the schemas for data exchanged between various components of a modern distributed system. Validating data being exchanged helps in maintaining data consistency and compatibility. +The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains schemas for data exchanged between various components of a modern distributed system. Validating exchanged data maintains data consistency and compatibility. -While several Schema Registry implementations exist, you will use the [Apicurio Registry](https://www.apicur.io/registry/) in this tutorial. Apicurio Registry is a popular open-source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. You will be using the Apicurio Registry in combination with Avro, which is a language-neutral data serialization system. +While several Schema Registry implementations exist, you will use the [Apicurio Registry](https://www.apicur.io/registry/) for this tutorial. Apicurio Registry is a popular open-source Schema Registry implementation that supports multiple serialization formats and facilitates schema management for diverse data in distributed systems. You will use Apicurio Registry combined with Avro, a language-neutral data serialization system. ## Prerequisites -Install [Docker](https://docs.docker.com/engine/install/) from the official website. +[Install Docker](https://docs.docker.com/engine/install/) from the official website. -## Steps ### Start Apicurio Registry -1. Start Apicurio Registry locally using below docker command: +Start the Apicurio Registry locally with the following docker command: ``` {`docker run --env CORS_ALLOWED_ORIGINS=* -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final`} ``` ### Upload Avro Schema -2. Once your local instance of Apicurio Registry is running, you can upload your Avro schema to it. Open new terminal window and create an Avro schema artifact using the following command: +Once your local instance of Apicurio Registry is running, upload your Avro schema. Open a new terminal window and create an Avro schema artifact with the following command: ``` curl \ @@ -59,7 +58,7 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. ### Replace link using JSON pointer -3. An alternative to having the schema in a separate file as you learned in [Describe Kafka message payload using Avro Schema](/docs/tutorials/kafka/configure-kafka-avro) where you used the existing Avro schema in your AsyncAPI document, would be to store it in the registry. After uploading your Avro schema, remove the schema from your document, and a $ref that points to the url from the previous step. +One alternative is to keep your schema in a separate file, as you learned in the previous tutorial, [Describe Kafka message payload using Avro Schema](/docs/tutorials/kafka/configure-kafka-avro). After uploading your Avro schema, remove the schema from your AsyncAPI document and add a `$ref` pointing to the previous step's URL. ``` $ref: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp ``` @@ -97,7 +96,7 @@ components: ``` ## Summary -In this tutorial you’ve gained insights about managing Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. It is important to note that this approach is valid for various other schema types. +In this tutorial, you managed Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. The good news is that this approach is valid for various other schema types! ## Next steps -Now that you have learnt how to manage schemas, check out our tutorial for [bindings with Kafka](/docs/tutorials/kafka/bindings-with-kafka) to start sending messages between your services. +Now that you have learned how to manage schemas, check out the [bindings with Kafka tutorial](/docs/tutorials/kafka/bindings-with-Kafka) to start sending messages between your services. From 38c96dcec62c958732b3ed612676309b88004d9a Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Thu, 8 Feb 2024 18:24:49 -0800 Subject: [PATCH 34/43] Delete pages/docs/tutorials/kafka/_section.md --- pages/docs/tutorials/kafka/_section.md | 4 ---- 1 file changed, 4 deletions(-) delete mode 100644 pages/docs/tutorials/kafka/_section.md diff --git a/pages/docs/tutorials/kafka/_section.md b/pages/docs/tutorials/kafka/_section.md deleted file mode 100644 index 994e7b8144a..00000000000 --- a/pages/docs/tutorials/kafka/_section.md +++ /dev/null @@ -1,4 +0,0 @@ ---- -title: 'Kafka' -weight: 2 ---- \ No newline at end of file From ebcfb8e5f5e048e2e37d4162a0dcde469e222e43 Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Thu, 8 Feb 2024 18:25:16 -0800 Subject: [PATCH 35/43] Delete pages/docs/tutorials/kafka/index.md --- pages/docs/tutorials/kafka/index.md | 4 ---- 1 file changed, 4 deletions(-) delete mode 100644 pages/docs/tutorials/kafka/index.md diff --git a/pages/docs/tutorials/kafka/index.md b/pages/docs/tutorials/kafka/index.md deleted file mode 100644 index e204dbc49ce..00000000000 --- a/pages/docs/tutorials/kafka/index.md +++ /dev/null @@ -1,4 +0,0 @@ ---- -title: Tutorial-1 -weight: 20 ---- \ No newline at end of file From 538d531e3939364f5cd0c69f14b1517cc1cd56b0 Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Fri, 9 Feb 2024 19:34:30 -0800 Subject: [PATCH 36/43] fix page weight --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 7a1fef5576b..e9c55a88638 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -1,7 +1,7 @@ --- title: Managing schemas using Schema Registry description: A tutorial teaching how to manage schemas using Schema Registry. -weight: 50 +weight: 250 --- ## Introduction From d426e86f144f028fc41aef5afd1e250926e724bc Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Fri, 16 Feb 2024 18:56:03 +0530 Subject: [PATCH 37/43] modified command for starting apicurio registry --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index e9c55a88638..05269a5173d 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -20,7 +20,7 @@ While several Schema Registry implementations exist, you will use the [Apicurio Start the Apicurio Registry locally with the following docker command: ``` -{`docker run --env CORS_ALLOWED_ORIGINS=* -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final`} +docker run --env CORS_ALLOWED_ORIGINS='*' -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final ``` ### Upload Avro Schema From c75bb44d1ef54fa21bfeb99fca58e30287ac6935 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Sun, 25 Feb 2024 12:51:19 +0530 Subject: [PATCH 38/43] updated heading Co-authored-by: Lukasz Gornicki --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 05269a5173d..0c72e31c981 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -57,7 +57,7 @@ Download your Avro schema by visiting the following URL: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. -### Replace link using JSON pointer +### Update schema reference One alternative is to keep your schema in a separate file, as you learned in the previous tutorial, [Describe Kafka message payload using Avro Schema](/docs/tutorials/kafka/configure-kafka-avro). After uploading your Avro schema, remove the schema from your AsyncAPI document and add a `$ref` pointing to the previous step's URL. ``` $ref: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp From ca1a55496627f02e85fcee29f39ea3e55fd85436 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Sun, 3 Mar 2024 12:02:49 +0530 Subject: [PATCH 39/43] updated introduction to mention previous tutorial --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 0c72e31c981..f312f37055b 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -5,7 +5,7 @@ weight: 250 --- ## Introduction -In this tutorial, you'll learn how to manage schemas with Schema Registry. Managing schemas ensures a consistent data exchange across distributed systems, thus minimizing errors. +In the [previous tutorial](/docs/tutorials/kafka/configure-kafka-avro), you've learned how to add Avro Schemas to your AsyncAPI document. Now, you will learn how to save your schema in a central Schema Registry and reuse it from there. ## Background context The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains schemas for data exchanged between various components of a modern distributed system. Validating exchanged data maintains data consistency and compatibility. From db82383334687f2a998687494188efa0bd6d6ea4 Mon Sep 17 00:00:00 2001 From: Arya Gupta Date: Sun, 3 Mar 2024 12:07:08 +0530 Subject: [PATCH 40/43] added AsyncAPI document with avro schema --- .../managing-schemas-using-schema-registry.md | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index f312f37055b..0e2b674da63 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -15,6 +15,47 @@ While several Schema Registry implementations exist, you will use the [Apicurio ## Prerequisites [Install Docker](https://docs.docker.com/engine/install/) from the official website. +## AsyncAPI document with Avro Schema +The previous tutorial taught you how to write an AsyncAPI document for Kafka messages using the Avro Schema. Here's an example of what an AsyncAPI document fully equipped with Avro Schema looks like: +``` +asyncapi: 3.0.0 +info: + title: User Signup API + version: 1.0.0 + description: The API notifies you whenever a new user signs up in the application. +servers: + kafkaServer: + host: test.mykafkacluster.org:8092 + description: Kafka Server + protocol: kafka +operations: + onUserSignedUp: + action: receive + channel: + $ref: '#/channels/userSignedUp' +channels: + userSignedUp: + description: This channel contains a message per each user who signs up in our application. + address: user_signedup + messages: + userSignedUp: + $ref: '#/components/messages/userSignedUp' +components: + messages: + userSignedUp: + payload: + schemaFormat: 'application/vnd.apache.avro;version=1.9.0' + schema: + type: record + name: UserSignedUp + namespace: com.company + doc: User sign-up information + fields: + - name: userId + type: int + - name: userEmail + type: string +``` ### Start Apicurio Registry Start the Apicurio Registry locally with the following docker command: From 04bc8b13d463491d1ff07ed507c39808abfa38c4 Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Fri, 15 Mar 2024 16:51:03 -0700 Subject: [PATCH 41/43] grammar fixes --- .../kafka/managing-schemas-using-schema-registry.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 0e2b674da63..9933181b5f9 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -5,7 +5,7 @@ weight: 250 --- ## Introduction -In the [previous tutorial](/docs/tutorials/kafka/configure-kafka-avro), you've learned how to add Avro Schemas to your AsyncAPI document. Now, you will learn how to save your schema in a central Schema Registry and reuse it from there. +In the [previous Kafka and Avro configuration tutorial](/docs/tutorials/kafka/configure-kafka-avro), you learned how to add Apache Avro schemas to your AsyncAPI document. Now, you will learn how to save your schema in a central Schema Registry and reuse it. ## Background context The need for schema management has become increasingly prevalent to handle the evolving complexity of modern Event-Driven Architecture. A Schema Registry is a centralized service that stores and maintains schemas for data exchanged between various components of a modern distributed system. Validating exchanged data maintains data consistency and compatibility. @@ -16,7 +16,7 @@ While several Schema Registry implementations exist, you will use the [Apicurio [Install Docker](https://docs.docker.com/engine/install/) from the official website. ## AsyncAPI document with Avro Schema -The previous tutorial taught you how to write an AsyncAPI document for Kafka messages using the Avro Schema. Here's an example of what an AsyncAPI document fully equipped with Avro Schema looks like: +The previous tutorial taught you how to write an AsyncAPI document for Kafka messages using the Avro schema. Here's an example of what an AsyncAPI document fully equipped with Avro schema looks like: ``` asyncapi: 3.0.0 info: @@ -99,7 +99,7 @@ http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp. ### Update schema reference -One alternative is to keep your schema in a separate file, as you learned in the previous tutorial, [Describe Kafka message payload using Avro Schema](/docs/tutorials/kafka/configure-kafka-avro). After uploading your Avro schema, remove the schema from your AsyncAPI document and add a `$ref` pointing to the previous step's URL. +One alternative is to keep your schema in a separate file, as you learned in the previous tutorial, [describe Kafka message payload using Avro schema](/docs/tutorials/kafka/configure-kafka-avro). After uploading your Avro schema, remove the schema from your AsyncAPI document and add a `$ref` pointing to the previous step's URL. ``` $ref: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp ``` From c4e35f9b2ee521955e55bec10a924c9a36c6c735 Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Fri, 15 Mar 2024 16:53:55 -0700 Subject: [PATCH 42/43] grammar fixes --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 9933181b5f9..9fff7f5b94b 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -64,7 +64,7 @@ Start the Apicurio Registry locally with the following docker command: docker run --env CORS_ALLOWED_ORIGINS='*' -it -p 8080:8080 apicurio/apicurio-registry-mem:2.5.8.Final ``` -### Upload Avro Schema +### Upload Avro schema Once your local instance of Apicurio Registry is running, upload your Avro schema. Open a new terminal window and create an Avro schema artifact with the following command: ``` From ad78b4ee2602db31bf47222bf8ce80bab8238cd2 Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Fri, 15 Mar 2024 17:00:29 -0700 Subject: [PATCH 43/43] fix link --- .../tutorials/kafka/managing-schemas-using-schema-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md index 9fff7f5b94b..9339e424e7b 100644 --- a/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md +++ b/pages/docs/tutorials/kafka/managing-schemas-using-schema-registry.md @@ -140,4 +140,4 @@ components: In this tutorial, you managed Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications. The good news is that this approach is valid for various other schema types! ## Next steps -Now that you have learned how to manage schemas, check out the [bindings with Kafka tutorial](/docs/tutorials/kafka/bindings-with-Kafka) to start sending messages between your services. +Now that you have learned how to manage schemas, check out the [bindings with Kafka tutorial](/docs/tutorials/kafka/bindings-with-kafka) to start sending messages between your services.