Data-Pipelining

This repository contains the artifacts for the presentation on Data Pipelining by Team 1 on tools such as GNU Make, Luigi, DASK, Celery and Airflow.

Video recording for the presentation can be found on Blackboard under Tegrity classes

Videos:

The links for the videos that we have recorded to explain the demo better are given below

Airflow

Part 1

https://youtu.be/Qs02p3mh8m4

Part2

https://youtu.be/83gPOMr6NOE

Celery

https://youtu.be/XelpLUcpKLM

DASK

https://www.youtube.com/watch?v=x4HGbh8mJNQ&t

Luigi

https://www.youtube.com/watch?v=NxsVEj0NeH4&t

Make

https://www.youtube.com/watch?v=Pb7lJMeXczo&t