feature/mkdag

[bwalsh]

This PR is an experiment:

• adds command mkdag

Which:

• Given a schema defined DAG configuration
• Produces:
  • dags/[id]_dag.py
  • plugins/hooks/[id]_hook.py
  • config/[id]_config.json

The intent is for the user to make application logic changes to the plugins/hooks/[id]_hook.py, while keeping dags/[id]_dag.py limited to workflow topology.

$ mkdag --help
Usage: mkdag [OPTIONS]

  CLI tool to render an Airflow DAG from a YAML configuration file

Options:
  -c, --config PATH  Path to the YAML configuration file.  [required]
  -o, --output PATH  Directory for the generated DAG files [dags/,
                     plugins/hooks, config/]. Default is .
  --help             Show this message and exit.

Known issues:

• sensor config todo
• [task complete sensor] (inter )
• OpenLineage Airflow Provider more