Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Add support for Databricks 14.3 ML LTS #10661

Open
15 tasks
sameerz opened this issue Apr 4, 2024 · 1 comment
Open
15 tasks

[FEA] Add support for Databricks 14.3 ML LTS #10661

sameerz opened this issue Apr 4, 2024 · 1 comment
Assignees
@razajafri
Labels
epic Issue that encompasses a significant feature or body of work feature request New feature or request

Comments

@sameerz
Copy link
Collaborator

sameerz commented Apr 4, 2024 •
edited by razajafri
Loading

Is your feature request related to a problem? Please describe.
Add support to the plugin for Databricks 14.3 ML LTS

Describe the solution you'd like
Add a shim layer for Databricks 14.3 (Spark 3.5.0)

Describe alternatives you've considered
N/A

Additional context

Issues affecting Databricks 14.3 even without Delta Lake and Deletion Vectors

Issues needed for basic Delta Lake support without Deletion Vectors

Issues pertaining to Deletion Vectors Read Support

Issues pertaining to Deletion Vectors Write Support

Low Priority Issues
@sameerz sameerz added feature request New feature or request ? - Needs Triage Need team to review and classify labels Apr 4, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Apr 9, 2024
@razajafri razajafri self-assigned this Sep 4, 2024
@mythrocks mythrocks mentioned this issue Oct 7, 2024
Merged
@mythrocks
Copy link
Collaborator

I've added #10969 to this task list. This might not be a deal breaker for Databricks 14.3, really. But I'm adding it here for tracking.

@gerashegalov gerashegalov mentioned this issue Jan 22, 2025
Merged
gerashegalov added a commit that referenced this issue Jan 23, 2025
@gerashegalov
Configure 14.3 support at runtime [databricks] (#11997)
c82a2ea 
Contributes to #10661, implements #11745 for DBR 14.3

- Add switch spark.rapids.shims.spark350db143.enabled

<!--

Thank you for contributing to RAPIDS Accelerator for Apache Spark!

Here are some guidelines to help the review process go smoothly.

1. Please write a description in this text box of the changes that are
being
   made.

2. Please ensure that you have written units tests for the changes
made/features
   added.

3. If you are closing an issue please use one of the automatic closing
words as
noted here:
https://help.github.com/articles/closing-issues-using-keywords/

4. If your pull request is not ready for review but you want to make use
of the
continuous integration testing facilities please label it with `[WIP]`.

5. If your pull request is ready to be reviewed without requiring
additional
   work on top of it, then remove the `[WIP]` label (if present).

6. Once all work has been done and review has taken place please do not
add
features or make changes out of the scope of those requested by the
reviewer
(doing this just add delays as already reviewed code ends up having to
be
re-reviewed/it is hard to tell what is new etc!). Further, please avoid
rebasing your branch during the review process, as this causes the
context
of any comments made by reviewers to be lost. If conflicts occur during
review then they should be resolved by merging into the branch used for
   making the pull request.

Many thanks in advance for your cooperation!

-->

---------

Signed-off-by: Gera Shegalov <[email protected]>
@razajafri razajafri mentioned this issue Jan 30, 2025
Open
@sameerz sameerz added the epic Issue that encompasses a significant feature or body of work label Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@razajafri razajafri

Labels
epic Issue that encompasses a significant feature or body of work feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@liurenjie1024 @mattahrens @mythrocks @sameerz @razajafri