Skip to content

Commit

Permalink
Revert "Redirect old location module imports to standard provider (ap…
Browse files Browse the repository at this point in the history
…ache#43610)" (apache#43946)

There is a very interesting breaking change introduced in Python 3.11
that will likely mean that we should not use MetaPathFinder for old
standard provider classes redirection.

There was a change introduced in Python 3.11 that caused that
module might not be found in some cases (for example when
unit test patches the path) when the module is loaded as
a different module (i.e. a.b differs from sys.modules['a.b'])

This is tracked in CPython via:
python/cpython#117860

This causes standard operator's tests fail in Python 3.11 and
3.12 when the providers/tests/standard/test_module_redirect_finder.py
is executed before - i.e. the standard modules are loaded as
old modules.

This reverts commit 5de2e73.
  • Loading branch information
potiuk authored and ellisms committed Nov 13, 2024
1 parent 2018c33 commit 25cb40c
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 89 deletions.
38 changes: 0 additions & 38 deletions airflow/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
import os
import sys
import warnings
from importlib.abc import MetaPathFinder
from typing import TYPE_CHECKING

if os.environ.get("_AIRFLOW_PATCH_GEVENT"):
Expand Down Expand Up @@ -134,40 +133,3 @@ def __getattr__(name: str):
from airflow import plugins_manager

plugins_manager.ensure_plugins_loaded()


class ModuleRedirectFinder(MetaPathFinder):
def __init__(self, redirects: dict[str, str]):
self.redirects: dict[str, str] = redirects

def find_spec(self, old_module: str, path, target=None):
import importlib

if old_module in self.redirects:
new_module_name = self.redirects[old_module]
warnings.warn(
f"Module '{old_module}' is deprecated, Please import it from '{new_module_name}'.",
DeprecationWarning,
stacklevel=2,
)
return importlib.util.find_spec(new_module_name)
return None


module_redirects_references: dict[str, str] = {
"airflow.operators.python": "airflow.providers.standard.operators.python",
"airflow.operators.bash": "airflow.providers.standard.operators.bash",
"airflow.operators.datetime": "airflow.providers.standard.operators.datetime",
"airflow.operators.weekday": "airflow.providers.standard.operators.weekday",
"airflow.sensors.bash": "airflow.providers.standard.sensors.bash",
"airflow.sensors.date_time": "airflow.providers.standard.sensors.date_time",
"airflow.sensors.python": "airflow.providers.standard.sensors.python",
"airflow.sensors.time": "airflow.providers.standard.sensors.time",
"airflow.sensors.time_delta": "airflow.providers.standard.sensors.time_delta",
"airflow.sensors.weekday": "airflow.providers.standard.sensors.weekday",
"airflow.hooks.filesystem": "airflow.providers.standard.hooks.filesystem",
"airflow.hooks.package_index": "airflow.providers.standard.hooks.package_index",
"airflow.hooks.subprocess": "airflow.providers.standard.hooks.subprocess",
"airflow.utils.python_virtualenv": "airflow.providers.standard.utils.python_virtualenv",
}
sys.meta_path.append(ModuleRedirectFinder(module_redirects_references))
51 changes: 0 additions & 51 deletions providers/tests/standard/test_module_redirect_finder.py

This file was deleted.

0 comments on commit 25cb40c

Please sign in to comment.