Skip to content

Commit

Permalink
Apply suggestions
Browse files Browse the repository at this point in the history
  • Loading branch information
sh-soltanpour committed Dec 28, 2023
1 parent 499b6df commit b2b28db
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions 001-cdc/001---cdc.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,15 +80,15 @@ For the subscriber/consumer part, we can try using the [Confluent PostgreSQL sin


### Approach 3: Implement our publisher and subscriber in Go, using the PostgreSQL's logical replication
In this approach, we will implement something similar to Debezium server/engine, but using Golang libraries such as [wal2json](https://github.com/eulerto/wal2json) and [wal-g](https://github.com/wal-g/wal-g).
In this approach, we will implement something similar to Debezium server/engine, but using tools and libraries such as [wal2json](https://github.com/eulerto/wal2json) and [wal-g](https://github.com/wal-g/wal-g).
The steps in this approach would be:
1. Using Wal-g, we create a base backup of the source database and restore it to the target database. From this point, the target database must be in read-only mode for users other than our plugin user.
1. Using wal-g, we create a base backup of the source database and restore it to the target database. From this point, the target database must be in read-only mode for users other than our plugin user.
2. Using wal2json, we create a stream of transactions from the source database from the point that the base backup was created.
3. A consumer reads the stream and applies the transactions to the target database.
4. From some point, we determine that the target database is somehow in sync with the source database, by somehow, I mean that there are less than a threshold number of transactions in the queue. At this point, we can say that databases are 'ready' for migration.
5. When databases are 'ready', and the user confirms, we need to put the source database in read-only mode and the consumer should process the remaining transactions until nothing is remaining.
4. At one point, we try to determine whether the target database is in sync with the source database or not, i.e. there are less than a threshold number of transactions in the queue. At this point, we can say that databases are 'ready' for migration.
5. When databases are 'ready', and the user confirms, we need to put the source database in read-only mode and the consumers should process the remaining transactions until all of them are processed.

During the time that the consumer is applying the remaining transaction, and the source DB is in read-only mode, there might be some write queries coming to the database. I can think of two ways to handle these queries:
While the consumers are applying the remaining transactions, and the source DB is in read-only mode, and there might be some write queries coming to the database. I can think of two ways to handle these queries:
1. Reject the queries and return an error to the client.
This approach will cause a very short glitch and downtime, but according to the fact that we already know that very few transactions remain, this downtime will be very short.

Expand Down

0 comments on commit b2b28db

Please sign in to comment.