Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AIP-84: Migrating GET queued asset events for DAG to fastAPI #43934

Merged

Conversation

amoghrajesh
Copy link
Contributor

@amoghrajesh amoghrajesh commented Nov 12, 2024

related: #42370

Migrating the GET queued asset events for a DAG endpoint to fast API.

Setup:

  1. Created producer and consumer dags:
    image

  2. Paused one producer while running the other to generate queued events
    image

Responses:

  1. Legacy
    image

  2. FastAPI
    image

With time filtering:

  1. Legacy
    image

(Beyond creation time)
image

  1. FastAPI
    image

(Beyond creation time)
image

Swagger spec: (maintained under dag section)
image

image


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Nov 12, 2024
@amoghrajesh amoghrajesh self-assigned this Nov 13, 2024
@amoghrajesh amoghrajesh marked this pull request as ready for review November 13, 2024 08:29
@amoghrajesh amoghrajesh added the legacy api Whether legacy API changes should be allowed in PR label Nov 13, 2024
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions

airflow/api_fastapi/core_api/datamodels/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/datamodels/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
@amoghrajesh
Copy link
Contributor Author

amoghrajesh commented Nov 15, 2024

@pierrejeambrun FYI, I have pushed the changes for all the "resolved" comments. The rest are open for input from you.

EDIT: I kept the endpoint at public/dags.py cos if I move to assets, the path will become: /public/assets/{dag_id}/assets/queuedEvent due to the underlying router registration. I moved the datamodels and tests to assets.py however

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small suggestions / questions almost ready

airflow/api_fastapi/common/utils.py Outdated Show resolved Hide resolved
airflow/api_fastapi/core_api/routes/public/dags.py Outdated Show resolved Hide resolved
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As many of other PRs depends on this one, to try to make them move forward I added a few commits, I think it's ready

@pierrejeambrun pierrejeambrun merged commit 3917730 into apache:main Nov 16, 2024
62 checks passed
@pierrejeambrun pierrejeambrun deleted the AIP84-get-all-queued-events-for-dag branch November 16, 2024 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. legacy api Whether legacy API changes should be allowed in PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants