Skip to content
This repository has been archived by the owner on Sep 4, 2024. It is now read-only.

Provision to pass queue name (to execute in specified airflow worker) while constructing DatabricksWorkflowTaskGroup #59

Open
ckodeboina opened this issue Jan 19, 2024 · 0 comments

Comments

@ckodeboina
Copy link

Use case: Execute all the databricks tasks in dedicated airflow worker.

Problem: The tasks created by "_CreateDatabricksWorkflowOperator" is executing in default airflow worker as is no provision to pass queue value to "_CreateDatabricksWorkflowOperator".

Proposed solution:
Give provision to pass operator parameters or just operator queue while constructing DatabricksWorkflowTaskGroup. So, while internally constructing _CreateDatabricksWorkflowOperator, those parameters can be passed.

Please let me know if any information is required.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant