41348.significant.rst

Rename Dataset as Asset

• list of renamed objects

  • Rename module airflow.api_connexion.schemas.dataset_schema as airflow.api_connexion.schemas.asset_schema

    • Rename variable create_dataset_event_schema as create_asset_event_schema
    • Rename variable dataset_collection_schema as asset_collection_schema
    • Rename variable dataset_event_collection_schema as asset_event_collection_schema
    • Rename variable dataset_event_schema as asset_event_schema
    • Rename variable dataset_schema as asset_schema
    • Rename class TaskOutletDatasetReferenceSchema as TaskOutletAssetReferenceSchema
    • Rename class DagScheduleDatasetReferenceSchema as DagScheduleAssetReferenceSchema
    • Rename class DatasetAliasSchema as AssetAliasSchema
    • Rename class DatasetSchema as AssetSchema
    • Rename class DatasetCollection as AssetCollection
    • Rename class DatasetEventSchema as AssetEventSchema
    • Rename class DatasetEventCollection as AssetEventCollection
    • Rename class DatasetEventCollectionSchema as AssetEventCollectionSchema
    • Rename class CreateDatasetEventSchema as CreateAssetEventSchema

  • Move module airflow.datasets to airflow.sdk.definitions.asset

    • Rename class DatasetAlias as AssetAlias
    • Rename class DatasetAll as AssetAll
    • Rename class DatasetAny as AssetAny
    • Rename function expand_alias_to_datasets as expand_alias_to_assets
    • Rename class DatasetAliasEvent as AssetAliasEvent
      • Rename attribute dest_dataset_uri as dest_asset_uri
    • Rename class BaseDataset as BaseAsset
      • Rename method iter_datasets as iter_assets
      • Rename method iter_dataset_aliases as iter_asset_aliases
    • Rename class Dataset as Asset
      • Rename method iter_datasets as iter_assets
      • Rename method iter_dataset_aliases as iter_asset_aliases
    • Rename class _DatasetBooleanCondition as _AssetBooleanCondition
      • Rename method iter_datasets as iter_assets
      • Rename method iter_dataset_aliases as iter_asset_aliases

  • Rename module airflow.datasets.manager as airflow.assets.manager

    • Rename variable dataset_manager as asset_manager

    • Rename function resolve_dataset_manager as resolve_asset_manager

    • Rename class DatasetManager as AssetManager

      • Rename method register_dataset_change as register_asset_change
      • Rename method create_datasets as create_assets
      • Rename method register_dataset_change as notify_asset_created
      • Rename method notify_dataset_changed as notify_asset_changed
      • Rename method notify_dataset_alias_created as notify_asset_alias_created

  • Rename module airflow.models.dataset as airflow.models.asset

    • Rename class DatasetDagRunQueue as AssetDagRunQueue
    • Rename class DatasetEvent as AssetEvent
    • Rename class DatasetModel as AssetModel
    • Rename class DatasetAliasModel as AssetAliasModel
    • Rename class DagScheduleDatasetReference as DagScheduleAssetReference
    • Rename class TaskOutletDatasetReference as TaskOutletAssetReference
    • Rename class DagScheduleDatasetAliasReference as DagScheduleAssetAliasReference

  • Rename module airflow.api_ui.views.datasets as airflow.api_ui.views.assets

    • Rename variable dataset_router as asset_rounter

  • Rename module airflow.listeners.spec.dataset as airflow.listeners.spec.asset

    • Rename function on_dataset_created as on_asset_created
    • Rename function on_dataset_changed as on_asset_changed

  • Rename module airflow.timetables.datasets as airflow.timetables.assets

    • Rename class DatasetOrTimeSchedule as AssetOrTimeSchedule

  • Rename module airflow.serialization.pydantic.dataset as airflow.serialization.pydantic.asset

    • Rename class DagScheduleDatasetReferencePydantic as DagScheduleAssetReferencePydantic
    • Rename class TaskOutletDatasetReferencePydantic as TaskOutletAssetReferencePydantic
    • Rename class DatasetPydantic as AssetPydantic
    • Rename class DatasetEventPydantic as AssetEventPydantic

  • Rename module airflow.datasets.metadata as airflow.sdk.definitions.asset.metadata

  • In module airflow.jobs.scheduler_job_runner

    • and its class SchedulerJobRunner

      • Rename method _create_dag_runs_dataset_triggered as _create_dag_runs_asset_triggered
      • Rename method _orphan_unreferenced_datasets as _orphan_unreferenced_datasets

  • In module airflow.api_connexion.security

    • Rename decorator requires_access_dataset as requires_access_asset

  • In module airflow.auth.managers.models.resource_details

    • Rename class DatasetDetails as AssetDetails

  • In module airflow.auth.managers.base_auth_manager

    • Rename function is_authorized_dataset as is_authorized_asset

  • In module airflow.timetables.simple

    • Rename class DatasetTriggeredTimetable as AssetTriggeredTimetable

  • In module airflow.lineage.hook

    • Rename class DatasetLineageInfo as AssetLineageInfo

      • Rename attribute dataset as asset

    • In its class HookLineageCollector

      • Rename method create_dataset as create_asset
      • Rename method add_input_dataset as add_input_asset
      • Rename method add_output_dataset as add_output_asset
      • Rename method collected_datasets as collected_assets

  • In module airflow.models.dag

    • Rename function get_dataset_triggered_next_run_info as get_asset_triggered_next_run_info

    • In its class DagModel

      • Rename method get_dataset_triggered_next_run_info as get_asset_triggered_next_run_info

  • In module airflow.models.taskinstance

    • and its class TaskInstance

      • Rename method _register_dataset_changes as _register_asset_changes

  • In module airflow.providers_manager

    • and its class ProvidersManager

      • Rename method initialize_providers_dataset_uri_resources as initialize_providers_asset_uri_resources
      • Rename attribute _discover_dataset_uri_resources as _discover_asset_uri_resources
      • Rename property dataset_factories as asset_factories
      • Rename property dataset_uri_handlers as asset_uri_handlers
      • Rename property dataset_to_openlineage_converters as asset_to_openlineage_converters

  • In module airflow.security.permissions

    • Rename constant RESOURCE_DATASET as RESOURCE_ASSET

  • In module airflow.serialization.enums

    • and its class DagAttributeTypes

      • Rename attribute DATASET_EVENT_ACCESSORS as ASSET_EVENT_ACCESSORS
      • Rename attribute DATASET_EVENT_ACCESSOR as ASSET_EVENT_ACCESSOR
      • Rename attribute DATASET as ASSET
      • Rename attribute DATASET_ALIAS as ASSET_ALIAS
      • Rename attribute DATASET_ANY as ASSET_ANY
      • Rename attribute DATASET_ALL as ASSET_ALL

  • In module airflow.serialization.pydantic.taskinstance

    • and its class TaskInstancePydantic

      • Rename method _register_dataset_changes as _register_dataset_changes

  • In module airflow.serialization.serialized_objects

    • Rename function encode_dataset_condition as encode_asset_condition
    • Rename function decode_dataset_condition as decode_asset_condition

  • In module airflow.timetables.base

    • Rename class `_NullDataset` as `_NullAsset`

      • Rename method iter_datasets as iter_assets
      • Rename method iter_dataset_aliases as iter_assets_aliases

  • In module airflow.utils.context

    • Rename class LazyDatasetEventSelectSequence as LazyAssetEventSelectSequence

  • In module airflow.www.auth

    • Rename function has_access_dataset as has_access_asset

  • Rename configuration core.dataset_manager_class as core.asset_manager_class and core.dataset_manager_kwargs as core.asset_manager_kwargs

  • Rename example dags example_dataset_alias.py, example_dataset_alias_with_no_taskflow.py, example_datasets.py as example_asset_alias.py, example_asset_alias_with_no_taskflow.py, example_assets.py

  • Rename DagDependency name dataset-alias, dataset as asset-alias, asset

  • Rename context key triggering_dataset_events as triggering_asset_events

  • Rename resource key dataset-uris as asset-uris for providers amazon, common.io, mysql, fab, postgres, trino

  • In provider airflow.providers.amazon.aws

    • Rename package datasets as assets

      • In its module s3

        • Rename method create_dataset as create_asset
        • Rename method convert_dataset_to_openlineage as convert_asset_to_openlineage

    • and its module auth_manager.avp.entities
      • Rename attribute AvpEntities.DATASET as AvpEntities.ASSET
    • and its module auth_manager.auth_manager.aws_auth_manager
      • Rename function is_authorized_dataset as is_authorized_asset

  • In provider airflow.providers.common.io

    • Rename package datasets as assets

      • in its module file

        • Rename method create_dataset as create_asset
        • Rename method convert_dataset_to_openlineage as convert_asset_to_openlineage

  • In provider airflow.providers.fab

    • in its module auth_manager.fab_auth_manager
      • Rename function is_authorized_dataset as is_authorized_asset

  • In provider airflow.providers.openlineage

    • in its module utils.utils
      • Rename class DatasetInfo as AssetInfo
      • Rename function translate_airflow_dataset as translate_airflow_asset

  • Rename package airflow.providers.postgres.datasets as airflow.providers.postgres.assets

  • Rename package airflow.providers.mysql.datasets as airflow.providers.mysql.assets

  • Rename package airflow.providers.trino.datasets as airflow.providers.trino.assets

  • Add module airflow.providers.common.compat.assets

  • Add module airflow.providers.common.compat.openlineage.utils.utils

  • Add module airflow.providers.common.compat.security.permissions

• Types of change

  • [x] Dag changes
  • [x] Config changes
  • [ ] API changes
  • [ ] CLI changes
  • [ ] Behaviour changes
  • [ ] Plugin changes
  • [ ] Dependency changes
  • [ ] Code interface changes

• Migration rules needed

  • airflow config lint
    • [x] core.dataset_manager_class → core.asset_manager_class
    • [x] core.dataset_manager_kwargs → core.asset_manager_kwargs
  • ruff
    • AIR302
      • [ ] context key triggering_dataset_events → triggering_asset_events
      • [x] airflow.api_connexion.security.requires_access_dataset → airflow.api_connexion.security.requires_access_asset
      • [x] airflow.auth.managers.base_auth_manager.is_authorized_dataset → airflow.auth.managers.base_auth_manager.is_authorized_asset
      • [x] airflow.auth.managers.models.resource_details.DatasetDetails → airflow.auth.managers.models.resource_details.AssetDetails
      • [x] airflow.lineage.hook.DatasetLineageInfo → airflow.lineage.hook.AssetLineageInfo
      • [x] airflow.security.permissions.RESOURCE_DATASET → airflow.security.permissions.RESOURCE_ASSET
      • [x] airflow.www.auth.has_access_dataset → airflow.www.auth.has_access_dataset.has_access_asset
      • [x] airflow.datasets.DatasetAliasEvent
      • [x] airflow.datasets.Dataset → airflow.sdk.definitions.asset.Asset
      • [x] airflow.Dataset → airflow.sdk.definitions.asset.Asset
      • [x] airflow.datasets.DatasetAlias → airflow.sdk.definitions.asset.AssetAlias
      • [x] airflow.datasets.DatasetAll → airflow.sdk.definitions.asset.AssetAll
      • [x] airflow.datasets.DatasetAny → airflow.sdk.definitions.asset.AssetAny
      • [x] airflow.datasets.metadata → airflow.sdk.definitions.asset.metadata
      • [x] airflow.datasets.expand_alias_to_datasets → airflow.sdk.definitions.asset.expand_alias_to_assets
      • [x] airflow.datasets.manager.dataset_manager → airflow.assets.manager
      • [x] airflow.datasets.manager.resolve_dataset_manager → airflow.assets.resolve_asset_manager
      • [x] airflow.datasets.manager.DatasetManager → airflow.assets.AssetManager
      • [x] airflow.listeners.spec.dataset.on_dataset_created → airflow.listeners.spec.asset.on_asset_created
      • [x] airflow.listeners.spec.dataset.on_dataset_changed → airflow.listeners.spec.asset.on_asset_changed
      • [x] airflow.timetables.simple.DatasetTriggeredTimetable → airflow.timetables.simple.AssetTriggeredTimetable
      • [x] airflow.timetables.datasets.DatasetOrTimeSchedule → airflow.timetables.assets.AssetOrTimeSchedule
      • [x] airflow.providers.amazon.auth_manager.avp.entities.AvpEntities.DATASET → airflow.providers.amazon.auth_manager.avp.entities.AvpEntities.ASSET
      • [x] airflow.providers.amazon.aws.datasets.s3.create_dataset → airflow.providers.amazon.aws.assets.s3.create_asset
      • [x] airflow.providers.amazon.aws.datasets.s3.convert_dataset_to_openlineage → airflow.providers.amazon.aws.datasets.s3.convert_dataset_to_openlineage
      • [x] airflow.providers.amazon.aws.datasets.s3.sanitize_uri → airflow.providers.amazon.aws.assets.s3.sanitize_uri
      • [x] airflow.providers.common.io.datasets.file.convert_dataset_to_openlineage → airflow.providers.common.io.assets.file.convert_asset_to_openlineage
      • [x] airflow.providers.common.io.datasets.file.sanitize_uri → airflow.providers.common.io.assets.file.sanitize_uri
      • [x] airflow.providers.common.io.datasets.file.create_dataset → airflow.providers.common.io.assets.file.create_asset
      • [x] airflow.providers.google.datasets.bigquery.sanitize_uri → airflow.providers.google.assets.bigquery.sanitize_uri
      • [x] airflow.providers.google.datasets.gcs.create_dataset → airflow.providers.google.assets.gcs.create_asset
      • [x] airflow.providers.google.datasets.gcs.sanitize_uri → airflow.providers.google.assets.gcs.sanitize_uri
      • [x] airflow.providers.google.datasets.gcs.convert_dataset_to_openlineage → airflow.providers.google.assets.gcs.convert_asset_to_openlineage
      • [x] airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_dataset → airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset
      • [x] airflow.providers.openlineage.utils.utils.DatasetInfo → airflow.providers.openlineage.utils.utils.AssetInfo
      • [x] airflow.providers.openlineage.utils.utils.translate_airflow_dataset → airflow.providers.openlineage.utils.utils.translate_airflow_asset
      • [x] airflow.providers.postgres.datasets.postgres.sanitize_uri → airflow.providers.postgres.assets.postgres.sanitize_uri
      • [x] airflow.providers.mysql.datasets.mysql.sanitize_uri → airflow.providers.mysql.assets.mysql.sanitize_uri
      • [x] airflow.providers.trino.datasets.trino.sanitize_uri → airflow.providers.trino.assets.trino.sanitize_uri
      • [x] property airflow.providers_manager.ProvidersManager.dataset_factories → airflow.providers_manager.ProvidersManager.asset_factories
      • [x] property airflow.providers_manager.ProvidersManager.dataset_uri_handlers → airflow.providers_manager.ProvidersManager.asset_uri_handlers
      • [x] property airflow.providers_manager.ProvidersManager.dataset_to_openlineage_converters → airflow.providers_manager.ProvidersManager.asset_to_openlineage_converters
      • [x] class attribute airflow.lineage.hook.DatasetLineageInfo.dataset → airflow.lineage.hook.AssetLineageInfo.asset
      • [x] method airflow.datasets.manager.DatasetManager.register_dataset_change → airflow.assets.manager.AssetManager.register_asset_change
      • [x] method airflow.datasets.manager.DatasetManager.create_datasets → airflow.assets.manager.AssetManager.create_assets
      • [x] method airflow.datasets.manager.DatasetManager.notify_dataset_created → airflow.assets.manager.AssetManager.notify_asset_created
      • [x] method airflow.datasets.manager.DatasetManager.notify_dataset_changed → airflow.assets.manager.AssetManager.notify_asset_changed
      • [x] method airflow.datasets.manager.DatasetManager.notify_dataset_alias_created → airflow.assets.manager.AssetManager.notify_asset_alias_created
      • [x] method airflow.providers.amazon.auth_manager.aws_auth_manager.AwsAuthManager.is_authorized_dataset → airflow.providers.amazon.auth_manager.aws_auth_manager.AwsAuthManager.is_authorized_asset
      • [x] method airflow.lineage.hook.HookLineageCollector.create_dataset → airflow.lineage.hook.HookLineageCollector.create_asset
      • [x] method airflow.lineage.hook.HookLineageCollector.add_input_dataset → airflow.lineage.hook.HookLineageCollector.add_input_asset
      • [x] method airflow.lineage.hook.HookLineageCollector.add_output_dataset → airflow.lineage.hook.HookLineageCollector.dd_output_asset
      • [x] method airflow.lineage.hook.HookLineageCollector.collected_datasets → airflow.lineage.hook.HookLineageCollector.collected_assets
      • [x] method airflow.providers_manager.ProvidersManager.initialize_providers_dataset_uri_resources → airflow.providers_manager.ProvidersManager.initialize_providers_asset_uri_resources
      • [x] method airflow.secrets.base_secrets.BaseSecretsBackend.get_conn_uri → airflow.secrets.base_secrets.BaseSecretsBackend.get_conn_value
      • [x] method airflow.secrets.base_secrets.BaseSecretsBackend.get_connections → airflow.secrets.base_secrets.BaseSecretsBackend.get_connection
      • [x] method airflow.hooks.base.BaseHook.get_connections → airflow.hooks.base.BaseHook.get_connection
      • [x] method airflow.datasets.BaseDataset.iter_datasets → airflow.sdk.definitions.asset.BaseAsset.iter_assets
      • [x] method airflow.datasets.BaseDataset.iter_dataset_aliases → airflow.sdk.definitions.asset.BaseAsset.iter_asset_aliases