Kafka Application: aggregating trade statistics

Overview

The Kafka application consists of 3 nodes.

1. CSV Reader Application (producer)
2. Kafka Broker
3. Aggregator Application (consumer)

The nodes communicate as below,

CSV Reader Application <-> Kafka Broker <-> Aggregator Application

Source/installation

The source for all 3 components can be found in this multi-module Maven repository.

git clone https://github.com/seagullmouse/kafka.git
cd kafka
mvn clean install

Usage

1. Start the Kafka Broker (requires docker-compose)
   docker-compose -f docker-compose.yml up

2. Start the Aggregator Application
   java -jar aggregator/target/aggregator-0.0.1-SNAPSHOT.jar

3. Start the CSV Reader Application (pass the CSV filename as the only argument)
   java -jar reader/target/reader-0.0.1-SNAPSHOT.jar ~/Downloads/commodity_trade_statistics_data.csv

Logs

• CSV Reader Application Log
• Aggregator Application Log