The platform contains the following components as described by the architecture diagram.
- You go from deploying a model to deploying a pipeline that automatically trains models.
- It allows to accelerate the experimentation of different models (e.g., trained on different data).
- Operational symmetry: The same pipeline (same code) can be used in both production and development environments.
- Forces modularization of pipeline components and allows them to be developed and replaced independently.
This project largely uses MLflow, which enables the management of the ML life cycle, from iteration on model development up to deployment in a reliable and scalable environment. https://www.mlflow.org/docs/latest/index.html
Apache Airflow is used to orchestrate the ML pipelines.
https://airflow.apache.org/docs/apache-airflow/stable/
To start the automated process:
make