Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: _get_external_task_sensor #21

Merged
merged 4 commits into from
Jul 17, 2023
Merged

Conversation

claudiazi
Copy link

@claudiazi claudiazi commented Jul 14, 2023

Issue:

The external sensor is poking the wrong external tag when checking the status of the external dag.

At the moment, we are using

from_pipeline_name = external_sensor_name.split("-")[0]
from_task_name = external_sensor_name.split("-")[1]

to define the from_pipeline_name and from_task_name to create the external sensor object.

However, there might be multiple - in the external_sensor_name. For example: core-salesforce-account_dimension_updater-sensor, with the current way, we will get the wrong from_pipeline_name and from_task_name which will create a sensor linked to a wrong/ non-existing external task.

Solution:

  • Get the sensor related params directly from the graph and export it as a dictionary
  • Use the params from this dictionary when creating the sensor instead of inferring from the sensor name

Test

Tested in: Airflow tst

@claudiazi claudiazi requested a review from a team as a code owner July 14, 2023 14:52
@pull-request-size pull-request-size bot added size/M and removed size/S labels Jul 14, 2023
@claudiazi claudiazi merged commit 86e020d into master Jul 17, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants