README.md

SmartcloudService

Pipecloud Group is a commercial cloud business intelligence provider Our goal is:

1. Be able to process the given data sets in batch 2. find proper cloud services to manage the essential tasks of computing, networking, storage and machine learning.

Our work:

1. Data sorce management (Spark & Snowflake)
2. Data cleaning and visualization (Amazon Kinesis流数据处理平台)
3. Machine learning pipeline (AWS EMR)
4. ELT pipeline and approach (AWS)
5. Result output and visualization (API & Tableau)

Tools:

managed by Pipecloud Group: Team member: Alex, Rex, Ryan, Alvin, Ning Ni, Niu Yue, Luna use this tool to create pipeline diagram: https://app.diagrams.net/

Project data pipeline: