Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata field to Kafka source #822

Merged
merged 1 commit into from
Jan 15, 2025
Merged

Add metadata field to Kafka source #822

merged 1 commit into from
Jan 15, 2025

Conversation

mwylde
Copy link
Member

@mwylde mwylde commented Jan 15, 2025

Adds key as a metadata field to the kafka source, allowing the message key to be injected into queries:

create table kafka (
    key BYTEA GENERATED ALWAYS AS (metadata('key')) STORED,
    topic TEXT GENERATED ALWAYS AS (metadata('topic')) STORED,
    value TEXT
)  with (
    connector = 'kafka',
    bootstrap_servers = 'localhost:9092',
    topic = 'logs',
    type = 'source',
    format = 'raw_string'
);

select cast(key as TEXT), value, topic from kafka;

@mwylde mwylde enabled auto-merge (squash) January 15, 2025 01:36
@mwylde mwylde merged commit 301af69 into master Jan 15, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant