Upgrade databricks provider dependency

[dcmshi]

continuation of #42626
patching some of the mypy test failures

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: [email protected]
  Slack: https://s.apache.org/airflow-slack

[dcmshi]

provider checks are good locally

hatch run pre-commit run mypy-providers --all-files

[dcmshi]

reran with manual mypy-providers check

hatch run pre-commit run --color always --verbose --hook-stage manual  mypy-providers --all-files

[dcmshi]

just syncing my fork, but the manual checks were passing before this so I think this should be ready for review?

[potiuk]

Approved the workflow: 👀

[dcmshi]

sorry I think I needed to rebase again, there were quite a few test failures in the test_backfill_endpoint module, but I think not related to my changes, I do see it was removed in this PR: https://github.com/apache/airflow/pull/43649/files . Could you please help me trigger checks whenever you get chance again 🙇

CI run test failures: https://github.com/apache/airflow/actions/runs/11671655535/job/32506671351?pr=43272

[eladkal]

How are changes to snowflake tests related to this PR?

[dcmshi]

The Snowflake test module is using the Row class from databricks.sql.types[0], which triggered the mypy checks on the module when trying to upgrade the databricks-sql-connecter package. I just went and patched the errors mypy was flagging, since I think refactoring the module to not use the Row class is probably it's own change there, but open to suggestions if this isn't the way we'd like to go.

[0] https://github.com/apache/airflow/pull/43272/files/175059dfe5cfb1f64297a8e26bcfe278edd7ebfb#diff-6fa19a213d772197807bb12413c4731e6b8a001a241f7f9009e152a1aac1f066R33

[eladkal]

Wait. What??
I am not near my laptop so I can't investigate this right now but this sounds very odd. We need to look into the commit that added it to see what was the reason for creating such odd coupling.

[dcmshi]

I think this is the original PR which introduces the import[0], and then this PR adds logic to mock the import later[1]. It looks to be testing the return of a SQLExecuteQueryOperator but the test_exec_success doesn't seem to be a Snowflake specific test?

[0] #28006
[1] #38074

[eladkal]

I dont think this is right.
Snowflake tests shuld not use databricks SDK

[potiuk]

Yeah. I think this is a mistake someone used databricks row in the snowflake tests - they "match" as they represent the output of dbapi that has generally similar structure, but this should be, I think fixed to have either some generic Row structure used or creating of own Row class in the tests.

[potiuk]

I think refactoring the module to not use the Row class is probably it's own change there, but open to suggestions if this isn't the way we'd like to go.

Yes. That's the way to go. Likely what would be great is to add similar tests that we have in snowflake now to databricks tests if possible, using databricks row, but to change the snowflake one to not use the databricks row