Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure HttpHook's auth_type from Connection #35591

Open
wants to merge 163 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
163 commits
Select commit Hold shift + click to select a range
5c905b2
feat: Implement `auth_kwargs` parameter in Http Connection
joffreybienvenu-infrabel Nov 12, 2023
1c7fcab
feat: Make auth_type configurable from Connection
joffreybienvenu-infrabel Nov 23, 2023
4d43f63
fix: Correctly use auth_type from Connection
joffreybienvenu-infrabel Nov 23, 2023
d5d3b3e
feat: Add Connection documentation
joffreybienvenu-infrabel Nov 24, 2023
e9848f5
feat: Make available auth_types configurable from airflow config
joffreybienvenu-infrabel Dec 5, 2023
6580407
feat: Add fields for auth config and header config in Http Connection…
joffreybienvenu-infrabel Dec 6, 2023
8203856
fix: Correctly apply styling to extra fields
joffreybienvenu-infrabel Dec 7, 2023
70f92a7
feat: Implement simplistic collapsable textarea for "extra"
joffreybienvenu-infrabel Dec 19, 2023
db2a859
fix: express clearly empty frozenset creation
joffreybienvenu-infrabel Dec 20, 2023
d14c80f
feat: Refactor Accordion TextArea to use wtform utils
joffreybienvenu-infrabel Dec 20, 2023
49c692e
feat: Implement 'collapse_extra' field behavior
joffreybienvenu-infrabel Dec 21, 2023
09e5057
feat: Implement parameterizable behavior for collapsible field
joffreybienvenu-infrabel Dec 30, 2023
2c11e76
feat: Deprecate headers params passed directly in 'Extra' field
joffreybienvenu-infrabel Dec 30, 2023
da4ef3b
revert: Remove collapsible field
joffreybienvenu-infrabel Jan 5, 2024
7393d26
feat: add 'Extra' deprecation warning in Connection UI
joffreybienvenu-infrabel Jan 5, 2024
6e98e4d
fix: set the default value for "auth_type" as empty string
joffreybienvenu-infrabel Jan 5, 2024
5306499
fix: Use Livy hook to test invalid extra removal
joffreybienvenu-infrabel Jan 5, 2024
b07a8cf
feat: Implement CodeMirrorField for providers
joffreybienvenu-infrabel Jan 8, 2024
53672f0
revert: Remove CodeMirror from providers
joffreybienvenu-infrabel Jan 8, 2024
099b9ad
feat: Add documentation
joffreybienvenu-infrabel Jan 8, 2024
35a26d8
feat: Implement auth_type and auth_kwargs in the AsyncHttpHook
joffreybienvenu-infrabel Jan 9, 2024
26bf63f
feat: Add tests
joffreybienvenu-infrabel Jan 9, 2024
bf7fe2f
fix: Add header and auth into FakeSession test object
joffreybienvenu-infrabel Jan 9, 2024
3130e05
fix: Use default BasicAuth in LivyAsyncHook
joffreybienvenu-infrabel Jan 9, 2024
709ef41
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
davidblain-infrabel Apr 8, 2024
80e1d49
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 8, 2024
2ed90b7
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 8, 2024
3f49edf
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 8, 2024
de77a7e
fix: Forgot to put self before invoking _url_from_endpoint
davidblain-infrabel Apr 9, 2024
1b65b4d
refactor: Removed json_safe_loads method as suggested by Joffrey and …
davidblain-infrabel Apr 9, 2024
09c5f5b
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 9, 2024
83f6648
refactor: Fixed some static checks in http provider
davidblain-infrabel Apr 9, 2024
2e5945b
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 9, 2024
6b8b71b
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 9, 2024
c6dd1c3
refactor: Moved _url_from_endpoint method to HttpHookMixin
davidblain-infrabel Apr 9, 2024
0c02a96
refactor: Moved default_auth_type to HttpHookMixin
davidblain-infrabel Apr 9, 2024
084f2f7
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 9, 2024
f8ac5e1
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 9, 2024
78f0475
refactor: Removed print statements and unused json param in AsyncHttp…
davidblain-infrabel Apr 9, 2024
0a7defd
refactor: Updated url_from_endpoint back to same implementation as in…
davidblain-infrabel Apr 9, 2024
0e1c159
refactor: Removed docstring for removed json parameter in run method …
davidblain-infrabel Apr 9, 2024
2bafed7
refactor: Aligned HttpTrigger with version from main branch
davidblain-infrabel Apr 9, 2024
e09af70
refactor: Removed test_trigger_on_post_with_data from TestHttpTrigger…
davidblain-infrabel Apr 9, 2024
7065713
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 9, 2024
522e3f2
refactor: Changed docstrings in HttpTrigger to imperative mode
davidblain-infrabel Apr 9, 2024
076c82f
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 9, 2024
ce05547
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 10, 2024
4711cc5
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 10, 2024
ede4241
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 11, 2024
050747c
refactor: Updated docstrings of serialize and run method of HttpTrigger
davidblain-infrabel Apr 11, 2024
82bcfd2
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 11, 2024
c0d1263
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 11, 2024
9b46125
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 11, 2024
a7716ee
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 11, 2024
fb60fc1
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 12, 2024
9020eb7
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 12, 2024
fc96332
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 12, 2024
6f985b8
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 15, 2024
9f01e64
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 23, 2024
02c5a00
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 25, 2024
4bcbe75
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Apr 26, 2024
e8fd487
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 3, 2024
b921e5d
fix: Fixed HTTP tests
davidblain-infrabel May 3, 2024
5919523
fix: Fixed static checks on HTTP tests
davidblain-infrabel May 3, 2024
2c24754
refactor: Mvoed get_connection_form_widgets method from HttpHookMixin…
davidblain-infrabel May 3, 2024
513bdad
refactor: Changed auth_type field in Http connection ui to string fie…
davidblain-infrabel May 6, 2024
fb2d2c2
refactor: Added HTTPKerberosAuth as possible AUTH_TYPE in HttpHook
davidblain-infrabel May 6, 2024
e1b7e5c
refactor: Moved get_connection_form_widgets method from HttpHook to H…
davidblain-infrabel May 6, 2024
c50ed8c
refactor: Pass auth_type parameter from LivyHook to constructor of Ht…
davidblain-infrabel May 6, 2024
a3cd370
refactor: Added other remaining connection properties in get_connecti…
davidblain-infrabel May 7, 2024
026fa31
refactor: Updated Javascript to render all textarea's in connection w…
davidblain-infrabel May 7, 2024
af246ad
refactor: Added debug logging statement and fixed error logging state…
davidblain-infrabel May 7, 2024
4669d46
refactor: Updated http tests changed some headers with real key/values
davidblain-infrabel May 7, 2024
7ca9393
refactor: Refactored some assertions in http tests
davidblain-infrabel May 7, 2024
72f9fbe
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 7, 2024
3c5246c
refactor: Reformatted Javascript like expected by static checks
davidblain-infrabel May 7, 2024
c15c705
refactor: Reformatted HTTP unit tests like expected by static checks
davidblain-infrabel May 7, 2024
1a41944
refactor: Re-added auth_type to HttpHookMixin
davidblain-infrabel May 7, 2024
4f3d2ab
refactor: Enhanced extra_dejson property to allow load string escaped…
davidblain-infrabel May 7, 2024
b7382f1
fix: Make sure value if set even if nested string structure can't be …
davidblain-infrabel May 7, 2024
e58df22
refactor: Fixed 2 additional static check remarks
davidblain-infrabel May 7, 2024
2ed5ab0
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 8, 2024
8bc23f0
refactor: Added test for extra_dejson property of Connection
davidblain-infrabel May 8, 2024
fc4d0fa
fix: Fixed the test of extra_dejson method on Connection
davidblain-infrabel May 8, 2024
b80cd88
refactor: Do not apply the CodeMirror on the description textarea
davidblain-infrabel May 8, 2024
27cf2e8
refactor: editor variable can also be a const
davidblain-infrabel May 8, 2024
227f90e
refactor: Applied 2 static checks changes
davidblain-infrabel May 8, 2024
435995c
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 8, 2024
d8a0c2e
refactor: Removed const from editor
davidblain-infrabel May 8, 2024
6a6aee9
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 8, 2024
8896766
refactor: Changed conn_type to ftp in test_process_form_invalid_extra…
davidblain-infrabel May 8, 2024
d881b09
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 8, 2024
106fdd0
refactor: Extracted get_extra_dejson method from extra_dejson propert…
davidblain-infrabel May 8, 2024
e70c70a
refactor: Changed docstring of get_extra_dejson method in imperative …
davidblain-infrabel May 8, 2024
162ff42
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 8, 2024
3b0f7d7
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 9, 2024
32fda3c
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 13, 2024
8c5ac06
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 13, 2024
10bd16e
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 14, 2024
70ee492
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla May 15, 2024
76f74f2
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jun 19, 2024
b6ba4d1
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jun 20, 2024
2667681
refactor: Updated HttpHook
davidblain-infrabel Jul 18, 2024
fe05aaf
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 18, 2024
81919bc
refactor: Reformatted some docstrings in HttpHook
davidblain-infrabel Jul 18, 2024
078d75e
refactor: HttpHook now uses patched version of Connection + added tes…
davidblain-infrabel Jul 18, 2024
27cf3e7
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 18, 2024
d1da151
refactor: Fixed some static checks
davidblain-infrabel Jul 18, 2024
94ad512
refactor: Fixed json import HttpHook
davidblain-infrabel Jul 18, 2024
e9278cb
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 19, 2024
1a1606c
docs: Updated docstrings
davidblain-infrabel Jul 19, 2024
ecc4619
docs: Fixed docstring ConnectionWithExtra
davidblain-infrabel Jul 19, 2024
87f9360
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 19, 2024
9859251
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 19, 2024
daf848e
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 19, 2024
43fad6f
docs: Added newline after docstring constructor
davidblain-infrabel Jul 19, 2024
00043c1
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 20, 2024
fdc1cbc
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jul 31, 2024
91f6fad
docs: Quoted the get_extra_dejson method in docstring to avoid spelli…
davidblain-infrabel Aug 1, 2024
e01c7a6
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 1, 2024
4d8e992
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 2, 2024
5f54c91
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 7, 2024
c9fe807
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 7, 2024
931f4c1
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 8, 2024
976b34d
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 8, 2024
a1b1418
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 9, 2024
52b9fde
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Aug 13, 2024
68479aa
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Sep 3, 2024
2b90331
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Sep 4, 2024
5c79ea1
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Sep 4, 2024
82a4a02
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 8, 2024
3ff766b
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
davidblain-infrabel Oct 10, 2024
cb3722a
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 10, 2024
378c2d8
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 10, 2024
2260576
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 10, 2024
04a57d9
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 11, 2024
c7fcba2
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 18, 2024
8f7c1c3
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 22, 2024
815e7d1
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Oct 23, 2024
b9afd30
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Nov 7, 2024
8119295
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Nov 12, 2024
9eb9325
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Nov 13, 2024
983785b
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Nov 19, 2024
c9c2ec4
refactor: Refactored HttpHook without mixin to have less difference w…
davidblain-infrabel Jan 10, 2025
ce066f0
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 10, 2025
c9bfc2c
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 10, 2025
b45de87
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 10, 2025
8e1d9eb
refactor: Reformatted imports HttpHook
davidblain-infrabel Jan 10, 2025
8d02ea9
refactor: Re-added missing except in HttpHook
davidblain-infrabel Jan 10, 2025
03a8bec
refactor: Fixed indentation TestHttpHook
davidblain-infrabel Jan 10, 2025
c20f7bd
refactor: Changed checks related to Airflow dependency for http provider
davidblain-infrabel Jan 10, 2025
dfcd97b
refactor: Reformatted TestHttpHook
davidblain-infrabel Jan 11, 2025
3b0ff43
refactor: Refactored test_airflow_dependency_version in TestHttpHook
davidblain-infrabel Jan 11, 2025
ef9f00a
refactor: Removed default_auth_type from LivyHook
davidblain-infrabel Jan 11, 2025
1a60887
refactor: Reformatted and removed duplicate _set_base_url method from…
davidblain-infrabel Jan 11, 2025
d14ec5b
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 11, 2025
f57fe39
refactor: Reorganized imports TestHttpHook
davidblain-infrabel Jan 11, 2025
6f102c0
refactor: Moved import AuthBase to type checking block
davidblain-infrabel Jan 14, 2025
48578ea
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 14, 2025
608f8ef
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 15, 2025
127ab40
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 17, 2025
b8c21de
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 17, 2025
077a606
Merge branch 'main' into feature/http-extra-auth-parameter-in-connection
dabla Jan 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions airflow/providers/apache/livy/hooks/livy.py
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,7 @@ def __init__(
extra_headers: dict[str, Any] | None = None,
) -> None:
super().__init__(http_conn_id=livy_conn_id)
self.auth_type = self.default_auth_type
self.extra_headers = extra_headers or {}
self.extra_options = extra_options or {}

Expand Down
Loading
Loading