-
Notifications
You must be signed in to change notification settings - Fork 14.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add "@asset" to decorate a function as a DAG and an asset #41325
Conversation
f0ea15c
to
017cdd9
Compare
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. |
91fb531
to
b6bb01b
Compare
2b7dbda
to
cc11458
Compare
@uranusjr I tried to make the |
6257830
to
0ef189e
Compare
3ac1a6a
to
a7e4dcc
Compare
…tor generate the right asset definition
…_fetch_active_assets_by_name
…inition instead of asset decorator
The validate_identifier validator already checks the length, so we don't need an extra one doing that.
0417860
to
2060ead
Compare
GitHub seems to have changed something and now I can’t approve my own PR. So consider this as a manual approval. ✅ |
Thanks! Let me merge it now |
They are really experimenting with Approvals it seems...... |
Why
As part of AIP-75
What
Add an
asset
decorator which can be used to decorate a function and create a DAG and an asset.In the example above, it create a
DAG(dag_id="asset2_producer", schedule=None, ...)
and anAsset(name="asset2_producer", uri="s3://bucket/object")
. Inside the function,self
can be used to access the asset with nameasset2_producer
,context
is just likecontext
in a normal task andasset1_producer
is used to access an asset with name=asset1_producerCloses: #42314