Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration/Data Migration: Add item about AWS DMS #115

Closed
wants to merge 1 commit into from
Closed

Integration/Data Migration: Add item about AWS DMS #115

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/_include/links.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
[Amazon DynamoDB Streams]: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html
[AWS DMS Kinesis Relay]: https://cratedb-toolkit.readthedocs.io/io/dms/index.html
[BM25]: https://en.wikipedia.org/wiki/Okapi_BM25
[cloud-datashader-colab]: https://colab.research.google.com/github/crate/cratedb-examples/blob/amo/cloud-datashader/topic/timeseries/explore/cloud-datashader.ipynb
[cloud-datashader-github]: https://github.com/crate/cratedb-examples/blob/amo/cloud-datashader/topic/timeseries/explore/cloud-datashader.ipynb
Expand Down
35 changes: 27 additions & 8 deletions docs/integrate/cdc/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,32 @@ to use them optimally.
Please also have a look at support for [generic ETL](#etl) solutions.
:::


## Amazon DynamoDB
:::{div}
Tap into [Amazon DynamoDB Streams], to replicate CDC events from DynamoDB into CrateDB,
with support for CrateDB's container data types.
- {hyper-open}`Documentation <[DynamoDB CDC Relay]>`
- {hyper-read-more}`Blog <[Replicating CDC events from DynamoDB to CrateDB]>`
:::

## AWS DMS
[AWS Database Migration Service] (AWS DMS) is a managed migration and replication
service that helps move your database and analytics workloads quickly, securely,
and with minimal downtime and zero data loss.

AWS DMS supports migration between 20-plus database and analytics engines, either
on-premises, or per EC2 instance databases. Supported data migration sources are:
Amazon Aurora, Amazon DocumentDB, Amazon S3, IBM DB2, MariaDB, Azure SQL Database,
Microsoft SQL Server, MongoDB, MySQL, Oracle, PostgreSQL, SAP ASE.

CrateDB provides two variants how to conduct data migrations using AWS DMS.
Either use it standalone / on your own premises, or use it in a completely
managed environment with services of AWS and CrateDB Cloud.

- {hyper-navigate}`AWS DMS Kinesis Relay (standalone) <[AWS DMS Kinesis Relay]>`
- {hyper-navigate}`AWS DMS Kinesis Relay (managed) <[AWS DMS Kinesis Relay]>`
Comment on lines +43 to +44
Copy link
Member Author

@amotl amotl Aug 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link targets are a work in progress. See:


## Debezium
Debezium is an open source distributed platform for change data capture (CDC).
It is built on top of Apache Kafka, a distributed streaming platform. It allows
Expand All @@ -30,14 +56,6 @@ SQL Server, IBM DB2, Cassandra, Vitess, Spanner, JDBC, and Informix.
- Webinar: [How to replicate data from other databases to CrateDB with Debezium and Kafka]
:::

## DynamoDB
:::{div}
Tap into [Amazon DynamoDB Streams], to replicate CDC events from DynamoDB into CrateDB,
with support for CrateDB's container data types.
- {hyper-open}`Documentation <[DynamoDB CDC Relay]>`
- {hyper-read-more}`Blog <[Replicating CDC events from DynamoDB to CrateDB]>`
:::

## MongoDB
:::{div}
Tap into [MongoDB Change Streams], to relay CDC events from MongoDB into CrateDB,
Expand All @@ -60,5 +78,6 @@ lives.



[AWS Database Migration Service]: https://aws.amazon.com/dms/
[How to replicate data from other databases to CrateDB with Debezium and Kafka]: https://cratedb.com/resources/webinars/lp-wb-debezium-kafka
[StreamSets Data Collector]: https://www.softwareag.com/en_corporate/platform/integration-apis/data-collector-engine.html
Loading