From f92932144760596dd820728ae209915164c6a36b Mon Sep 17 00:00:00 2001 From: Floor Drees Date: Fri, 8 Dec 2023 12:21:07 +0000 Subject: [PATCH 1/4] Add redirect to aiven.io/community --- _redirects | 1 + 1 file changed, 1 insertion(+) diff --git a/_redirects b/_redirects index 08564efddf..e2fe26e333 100644 --- a/_redirects +++ b/_redirects @@ -13,6 +13,7 @@ /api /docs/tools/api /cli /docs/tools/cli.html /terraform /docs/tools/terraform +/community https://aiven.io/community/forum/ # Renamed/deleted files /docs/products/flink/howto/real-time-alerting-solution-cli.html /docs/products/flink/howto/real-time-alerting-solution.html From 723d1845106e58ddec5609cd0a8b151818773484 Mon Sep 17 00:00:00 2001 From: Floor Drees Date: Fri, 8 Dec 2023 13:22:34 +0100 Subject: [PATCH 2/4] Update _redirects --- _redirects | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_redirects b/_redirects index e2fe26e333..70f8b259d6 100644 --- a/_redirects +++ b/_redirects @@ -13,7 +13,7 @@ /api /docs/tools/api /cli /docs/tools/cli.html /terraform /docs/tools/terraform -/community https://aiven.io/community/forum/ +/community https://aiven.io/community/ # Renamed/deleted files /docs/products/flink/howto/real-time-alerting-solution-cli.html /docs/products/flink/howto/real-time-alerting-solution.html From ecf485dcd3daf6df3173430d6fafdb9717943eae Mon Sep 17 00:00:00 2001 From: Floor Drees Date: Fri, 8 Dec 2023 14:45:42 +0000 Subject: [PATCH 3/4] remove community.rst --- docs/community.rst | 80 ---------------------------------------------- 1 file changed, 80 deletions(-) delete mode 100644 docs/community.rst diff --git a/docs/community.rst b/docs/community.rst deleted file mode 100644 index fc5045921f..0000000000 --- a/docs/community.rst +++ /dev/null @@ -1,80 +0,0 @@ -Aiven community -=============== - -There are lots of other ways to learn more about Aiven, and interact with us. - - -.. grid:: 1 2 2 2 - - .. grid-item-card:: - :shadow: md - :margin: 2 2 0 0 - - |icon-twitter| **Twitter** We love to chat! Tell us what you're building with Aiven! - - .. button-link:: https://twitter.com/aiven_io - :align: right - :color: primary - :outline: - - To the Twitterverse - - .. grid-item-card:: - :shadow: md - :margin: 2 2 0 0 - - |icon-blog| **Blog** To read tech news, tutorials, and updates on what we're up to. - - - .. button-link:: https://aiven.io/blog - :align: right - :color: primary - :outline: - - To the blog - - .. grid-item-card:: - :shadow: md - :margin: 2 2 0 0 - - |icon-youtube| **YouTube** If video is your thing, cool, it's ours too. - - .. button-link:: https://www.youtube.com/c/Aiven_io - :align: right - :color: primary - :outline: - - To YouTube - - .. grid-item-card:: - :shadow: md - :margin: 2 2 0 0 - - |icon-github| **GitHub** Find our public repositories on GitHub, work on `open source `_ with us! - - - .. button-link:: https://github.com/aiven - :align: right - :color: primary - :outline: - - To the code - - - -Check out the `Aiven Console `_ for your Aiven web interface. New accounts get a free trial! - ----------------- - -Documentation contributors --------------------------- - -We love our documentation so much that we created documentation about the documentation! If you want to learn more about the platform, or contribute some changes of your own, everything you need is in the :doc:`/docs/community/documentation` section. - -Mini challenges ---------------- - -Get to know Aiven through our mini challenges, each introducing you to different open source technologies. - -* :doc:`Catch the bus (Clickhouse) ` -* :doc:`Rolling challenge (Apache Kafka & Flink) ` \ No newline at end of file From edb59ebde39524625347b135103a41112056f505 Mon Sep 17 00:00:00 2001 From: ArthurFlageul Date: Fri, 8 Dec 2023 17:00:40 +0100 Subject: [PATCH 4/4] delete and fix build --- _redirects | 6 +- _toc.yml | 9 -- docs/community/challenge/catch-the-bus.rst | 26 ---- .../challenge/the-rolling-challenge.rst | 118 ------------------ 4 files changed, 4 insertions(+), 155 deletions(-) delete mode 100644 docs/community/challenge/catch-the-bus.rst delete mode 100644 docs/community/challenge/the-rolling-challenge.rst diff --git a/_redirects b/_redirects index 70f8b259d6..8bb461d612 100644 --- a/_redirects +++ b/_redirects @@ -74,13 +74,15 @@ /docs/tools/terraform/reference/cookbook/kafka-mongodb-recipe https://aiven.io/developer/apache-kafka-with-mongodb /docs/tools/terraform/reference/cookbook/kafka-debezium-postgres-source https://aiven.io/developer/debezium-source-postgresql-kafka-across-clouds /docs/tools/terraform/reference/cookbook/kafka-topics-http-connector-recipe https://aiven.io/developer/kafka-with-http-sink -/docs/tools/terraform/reference/cookbook/kafka-custom-conf-recipe https://aiven.io/developer/apache-kafka-with-custom-configurations +/docs/tools/terraform/reference/cookbook/kafka-custom-conf-recipe https://aiven.io/developer/apache-kafka-with-custom-configurations /docs/tools/terraform/reference/cookbook/m3db-m3agg-recipe https://aiven.io/developer/m3-aggregator-integration /docs/tools/terraform/reference/cookbook/postgresql-read-replica-recipe https://aiven.io/developer/postgresql-read-only-terraform /docs/tools/terraform/reference/cookbook/clickhouse-access-setup-recipe https://aiven.io/developer/manage-user-privileges-clickhouse-terraform /docs/products/clickhouse/howto/configure-access-terraform-deployed https://aiven.io/developer/manage-user-privileges-clickhouse-terraform /docs/tools/terraform/reference/cookbook/kafka-clickhouse-integration-recipe https://aiven.io/developer/kafka-source-for-clickhouse -/docs/tools/terraform/reference/cookbook/postgres-clickhouse-integration-recipe https://aiven.io/developer/postgresql-source-for-clickhouse +/docs/tools/terraform/reference/cookbook/postgres-clickhouse-integration-recipe https://aiven.io/developer/postgresql-source-for-clickhouse +/docs/community/challenge/catch-the-bus https://aiven.io/community/ +/docs/community/challenge/the-rolling-challenge https://aiven.io/community/ # Redirect from .index.html to specific page names for landing diff --git a/_toc.yml b/_toc.yml index dd295bca9b..2340fb0065 100644 --- a/_toc.yml +++ b/_toc.yml @@ -1353,12 +1353,3 @@ entries: title: Upgrade your support tier - file: docs/platform/concepts/service-level-agreement - file: docs/platform/concepts/beta_services - - # -------- COMMUNITY -------- - - file: docs/community - title: Community - entries: - - file: docs/community/challenge/catch-the-bus - title: Catch the Bus - Aiven challenge with ClickHouse - - file: docs/community/challenge/the-rolling-challenge - title: Rolling - Aiven challenge with Apache Kafka and Apache Flink diff --git a/docs/community/challenge/catch-the-bus.rst b/docs/community/challenge/catch-the-bus.rst deleted file mode 100644 index 115be50c0b..0000000000 --- a/docs/community/challenge/catch-the-bus.rst +++ /dev/null @@ -1,26 +0,0 @@ -Catch the bus - Aiven challenge with ClickHouse® -================================================ - -Welcome to Aiven's "Catch the bus" challenge, an easy way for you to explore Aiven for ClickHouse®. - -With the launch of Aiven for ClickHouse® we finally have the "C" for our KFC stack, the best combination of tools for real-time alerting and historic analytics. The stack utilises Apache Kafka®, Apache Flink®, and ClickHouse® to build up a robust, scalable architecture for getting the most from your data, whether it be batched ETL or real-time sensors. - -For this challenge, we'll be using real-time data from `Digitransit `_ to plot maps and find new insights. Digitransit provides real-time bus information and positioning data for busses in Helsinki. We've saved you a few weeks worth of data as a CSV to experiment with. - -Let's dive right in. - -Instructions ------------- -The goal is to find the best bus routes between the different KFC restaurants in Helsinki. - -1. Create an Aiven free trial account: `sign up for free `_. - -2. `Download the dataset `_. - -3. Spin up an Aiven for ClickHouse® service and load the dataset. Please see the `Aiven for ClickHouse® docs `_ for help with these steps. - -4. Find the routes and plot them on a map. Here's a `notebook `_ with a few optional queries and a mapping library to get you started. You can also check out the `ClickHouse docs `_ for advanced SQL information. - -5. If you find the solution or need help, email us at `hacks@Aiven.io `_. - -Good luck! \ No newline at end of file diff --git a/docs/community/challenge/the-rolling-challenge.rst b/docs/community/challenge/the-rolling-challenge.rst deleted file mode 100644 index c97dc30b8f..0000000000 --- a/docs/community/challenge/the-rolling-challenge.rst +++ /dev/null @@ -1,118 +0,0 @@ -Aiven "rolling challenge" with Apache Kafka® and Apache Flink® -============================================================== - -Welcome to Aiven's "Rolling" challenge, an easy way for you to explore Aiven for Apache Kafka® and Aiven for Apache Flink®. - -With the launch of Aiven for Apache Flink® we added a new way to manipulate your Apache Kafka® streaming data via SQL statements, providing the best combination of tools for real-time data transformation. - -For this challenge, we'll be using `Aiven fake data generator on Docker `_ to generate a series of symbols. The challenge consists of understanding the overall meaning of the symbols by transforming the original series of data with Apache Flink. - -Let's dive right in. - - -Instructions ------------- - -The goal is to make sense of the incoming stream of data. - - -1. Create an Aiven free trial account: `sign up for free `_. - -2. Create an :doc:`Aiven for Apache Kafka® ` and :doc:`Aiven for Apache Flink® ` service - -3. Set up an :doc:`integration between the Aiven for Apache Kafka® and Apache Flink® services ` - -4. Create a new :doc:`Aiven authentication token ` - -5. Clone the `Aiven fake data generator on Docker `_ with: - - .. code:: - - git clone https://github.com/aiven/fake-data-producer-for-apache-kafka-docker - -6. Copy the file ``conf/env.conf.sample`` to ``conf/env.conf`` and edit the following parameters - - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - | Parameter Name | Parameter Value | - +================+==============================================================================================================================+ - |PROJECT_NAME | Name of the Aiven Project where the Aiven for Apache Kafka service is running | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |SERVICE_NAME | Name of the Aiven for Apache Kafka service running | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |TOPIC | Name of the Topic to write messages in. ``rolling`` for the challenge | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |PARTITIONS | 5 | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |REPLICATION | 2 | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |NR_MESSAGES | 0 | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |MAX_TIME | 0 | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |SUBJECT | ``rolling`` | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |USERNAME | Your Aiven account username | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |TOKEN | Your Aiven account token | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |PRIVATELINK | ``NO`` | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - |SECURITY | ``SSL`` | - +----------------+------------------------------------------------------------------------------------------------------------------------------+ - -7. Build the Docker image - - .. code:: - - docker build -t fake-data-producer-for-apache-kafka-docker . - -8. Run the Docker image - - .. code:: - - docker run fake-data-producer-for-apache-kafka-docker - -9. Check the fake messages being produced by Docker - -10. In the `Aiven Console `_, navigate to the Aiven for Apache Flink service page - -11. Play with the Aiven for Apache Flink **Application** tab and try to make sense of the data. - - .. Tip:: - - The source table can be mapped in Aiven for Apache Flink with the following SQL, using the ``rolling`` topic as source: - - .. code:: - - - CREATE TABLE ROLLING_IN( - ts BIGINT, - val string, - ts_ltz AS TO_TIMESTAMP_LTZ(ts, 3), - WATERMARK FOR ts_ltz AS ts_ltz - INTERVAL '10' SECOND - ) - WITH ( - 'connector' = 'kafka', - 'properties.bootstrap.servers' = '', - 'topic' = 'rolling', - 'value.format' = 'json', - 'scan.startup.mode' = 'earliest-offset' - ) - - -12. When you find the solution, email a screenshot to hacks@Aiven.io - -Tips ----- - -Some tips that could help in solving the challenge: - -* ``kcat`` is a tool to explore data in Apache Kafka topics, check the :doc:`dedicate documentation ` to understand how to use it with Aiven for Apache Kafka -* ``jq`` is a helpful tool to parse JSON payloads, read `the instructions `_ on how to install and check the following useful flags: - * ``-r`` retrieves the raw output - * ``-j`` doesn't create a new line for every message - * ``-c`` shows data in compact view - -* If you're stuck with visualizing ``kcat`` consumer data with ``jq``, check the `-u` flag `as per dedicated example `_ - -