Skip to content

Commit

Permalink
Create aws-kafka.md (#624)
Browse files Browse the repository at this point in the history
* Create aws-kafka.md

* Rename aws-kafka.md to aws-kafka.md

* Update _meta.json

* Update integrations.mdx

* Update aws-kafka.md

---------

Co-authored-by: ishamehramixpanel <[email protected]>
Co-authored-by: myronkaifung <[email protected]>
  • Loading branch information
3 people authored Aug 15, 2024
1 parent 74a5d8f commit 77fecfb
Show file tree
Hide file tree
Showing 3 changed files with 60 additions and 0 deletions.
1 change: 1 addition & 0 deletions pages/docs/tracking-methods/integrations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ you're looking for [reach out to us](https://mixpanel.com/get-support).

<Cards>
<Card icon title="Amazon S3" href="/docs/tracking-methods/integrations/amazon-s3" />
<Card icon title="Amazon Kafka" href="/docs/tracking-methods/integrations/aws-kafka" />
<Card icon title="CMS & E-Commerce" href="/docs/tracking-methods/integrations/cms-ecommerce" />
<Card icon title="Customer.io" href="https://customer.io/docs/cdp/destinations/connections/mixpanel?utm_source=mixpanel&utm_medium=partner" />
<Card icon title="Freshpaint" href="/docs/tracking-methods/integrations/freshpaint" />
Expand Down
1 change: 1 addition & 0 deletions pages/docs/tracking-methods/integrations/_meta.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{
"amazon-s3": "Amazon S3",
"aws-kafka": "Amazon Kafka",
"cms-ecommerce": "CMS & E-Commerce",
"customer-io": {
"title": "Customer.io ↗",
Expand Down
58 changes: 58 additions & 0 deletions pages/docs/tracking-methods/integrations/aws-kafka.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Kafka

This guide demonstrates how to plug Mixpanel into an event collection pipeline by AWS Kafka. Once set up, your events will route to Mixpanel and be available in real-time for analytics. This approach is serverless and open-source, and takes ~5 minutes to set up.

To read messages from a Kafka topic and send them to Mixpanel, you'll need to use the confluent-kafka library for Kafka and the mixpanel library for Mixpanel.

### Here's a sample Python code snippet:

```python main.py
from confluent_kafka import Consumer, KafkaError
from mixpanel import Mixpanel

# Kafka setup
kafka_config = {
'bootstrap.servers': 'localhost:9092',
'group.id': 'my-group',
'auto.offset.reset': 'earliest'
}
consumer = Consumer(kafka_config)
consumer.subscribe(['my_topic'])

# Mixpanel setup
mp = Mixpanel('YOUR_MIXPANEL_TOKEN')

try:
while True:
msg = consumer.poll(1)

if msg is None:
continue
if msg.error():
if msg.error().code() == KafkaError._PARTITION_EOF:
continue
else:
print(msg.error())
break

# Send to Mixpanel
# The event_data contains the entire event. Consider extracting parsing the event for properties and explicitly assigning them instead.
event_data = msg.value().decode('utf-8')
mp.track('some_user_id', 'My Event', {'message': event_data})

finally:
consumer.close()

```


To run this code:

1. Install Kafka and start a Kafka broker.
2. Install necessary Python packages:
- pip install confluent-kafka
- pip install mixpanel
3. Replace 'YOUR_MIXPANEL_TOKEN' with your actual Mixpanel token.
4. Run the code.

Google has a [great reference](https://cloud.google.com/pubsub/docs/handling-failures) on best practices for handling both retryable and non-retryable errors.

0 comments on commit 77fecfb

Please sign in to comment.