Skip to content

Commit

Permalink
prettier fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
alexfrancoeur committed Jan 14, 2025
1 parent 78f0797 commit 0901bd9
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions postgres-cafe-pgstream.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ slug: postgres-cafe-pgstream
ogImage: https://raw.githubusercontent.com/xataio/mdx-blog/main/images/[email protected]
---

In [episode 3 of Postgres Café](https://www.youtube.com/embed/j1R3a0-jg6c?si=rEPoVI9vyR2MI_e8), we’re talking about pgstream, an open-source tool designed to handle one of PostgreSQL’s biggest gaps: seamlessly replicating schema and data changes.
In [episode 3 of Postgres Café](https://www.youtube.com/embed/j1R3a0-jg6c?si=rEPoVI9vyR2MI_e8), we’re talking about pgstream, an open-source tool designed to handle one of PostgreSQL’s biggest gaps: seamlessly replicating schema and data changes.

# Episode 3: pgstream

Expand All @@ -38,19 +38,22 @@ Schema changes, such as altering table structures or adding columns, can disrupt
### Tackling replication lag

pgstream prevents replication slot lag with two key approaches:

- **Memory buffering:** Allocates memory to keep consuming replication slots even when downstream systems are slow.
- **Kafka integration:** Decouples upstream and downstream systems by writing changes to Kafka topics, allowing multiple consumers to process data independently and at their own pace.

### Modular and extensible design

pgstream’s architecture is built around listeners (reading data changes) and processors (handling those changes). This modular design allows for flexible integrations and customizations. For example:

- Use a **Postgres-to-OpenSearch processor** for real-time search.
- Build custom processors to integrate with Snowflake, S3, or any other system.
- Leverage its webhook processor to trigger external workflows based on database events.

### Use cases

The team shared how pgstream is already in use at Xata:

- **Search integration:** pgstream powers Postgres-to-OpenSearch replication to enable advanced search features directly from PostgreSQL data.
- **File cleanup workflows:** A custom processor detects when files are deleted in Postgres and triggers corresponding deletions in S3 storage, keeping data consistent across systems.

Expand All @@ -62,4 +65,4 @@ For an in-depth exploration of pgstream and its capabilities, watch the full epi

## Stay tuned for more Postgres tools

This is part of the Postgres Cafe video series! [Subscribe to the playlist](https://www.youtube.com/playlist?list=PLf7KS0svgDP_zJmby3RMzzOVO45qLbruA) for more episodes that feature open-source tools like [pgroll](https://pgroll.com/) for zero downtime schema changes in production, [StatsMgr](https://www.data-bene.io/en/blog/postgres-cafe-expand-monitoring-capabilities-with-statsmgr/) for monitoring and tracking events across PostgreSQL, and more. Watch this space to learn how each tool can make working with Postgres smoother and more efficient.
This is part of the Postgres Cafe video series! [Subscribe to the playlist](https://www.youtube.com/playlist?list=PLf7KS0svgDP_zJmby3RMzzOVO45qLbruA) for more episodes that feature open-source tools like [pgroll](https://pgroll.com/) for zero downtime schema changes in production, [StatsMgr](https://www.data-bene.io/en/blog/postgres-cafe-expand-monitoring-capabilities-with-statsmgr/) for monitoring and tracking events across PostgreSQL, and more. Watch this space to learn how each tool can make working with Postgres smoother and more efficient.

0 comments on commit 0901bd9

Please sign in to comment.