An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Zookeeper docker container, ready for deployments on kubernetes and openshift

Dockerfiles & YAML definitions for Apache Drill

A distributed and highly available URL Shortener built on MERN stack, using Apache Zookeeper, cached using Redis, containerized by Docker.

Proof of concept open source Amazon Athena

Helm Charts to Deploy Apache Drill on Kubernetes

Thesis project: topic categorization and sentiment analysis on twitter with Apache Spark

Fully Managed Apache Zookeeper Cluster with Ansible & Terraform.

End-to-end data engineering pipeline with various technologies to ingest real time data.

This project aims to consume a twitter stream via Apache Kafka, apply a sentiment analysis on the tweets with Apache Spark jobs and save the result into Apache HBase.

A Visual Studio Code extension for managing Apache Zookeeper

Real time streaming of a time series with corresponding forecasts.

A Smart Traffic Management System for Ho Chi Minh City, Vietnam leveraging batch and real-time data processing, intuitive dashboards, and monitoring tools to optimize traffic flow, enhance safety, and support sustainable urban mobility through advanced analytics and user-friendly applications.

Apache Drill and Apache Zookeeper helm charts for kubernetes

A demonstration with Spring Batch framework, Kafka, and PostgreSQL

A real-time, dockerized Kafka event processor pipeline (utilizing an Apache Storm topology). The project is running in AWS with automated infrastructure deployment and execution, using Ansible.

An Ansible role which installs and configures Apache Zookeeper on Redhat/Debian based hosts

Distributed Java Applications at Scale, Parallel Programming, Distributed Computing

Docker image packaging for Apache Zookeeper