Constrain Task Callback to Specific Task Operator Types

[GabrielEisenbergOlympus]

Hi all 😄

I'm working with Airflow 2.10.1 on MWAA and need to send results from specific tasks to an external service. Based on certain constraints, this will need to be done in a callback. For ease of other users, I want to generalise this into standard callbacks that we use.

My question is whether there is a way to constrain these task callbacks to certain types of operators without including a logic check inside the callback itself? In this way Airflow would do the filtering and would only invoke the callback in certain cases and not for every single task.

Thanks so much!

[potiuk]

No. Not as a direct feature. But you could possibl do something with cluster policies https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/cluster-policies.html - you could likely analyse your task in task_policy, see if your callback defined is as expected (according to whatever criteria you choose) and either remove the callback or - possibly better - throw an exception which will fail the DAG to being created.