Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Build broken on Databricks 14.3 after #11996 #12074

Closed
gerashegalov opened this issue Feb 6, 2025 · 2 comments · Fixed by #12075
Closed

[BUG] Build broken on Databricks 14.3 after #11996 #12074

gerashegalov opened this issue Feb 6, 2025 · 2 comments · Fixed by #12075
Assignees
@gerashegalov
Labels
bug Something isn't working

Comments

@gerashegalov
Copy link
Collaborator

gerashegalov commented Feb 6, 2025 •
edited
Loading

Describe the bug
Build succeed without #11996
and fails with it

$ ./jenkins/databricks/build.sh
[INFO] Compiling 45 Scala sources to /home/ubuntu/spark-rapids/delta-lake/delta-spark350db143/target/spark350db143/classes...
[ERROR] /home/ubuntu/spark-rapids/delta-lake/common/src/main/databricks/scala/org/apache/spark/sql/rapids/delta/GpuOptimizeWriteExchangeExec.scala:91: not enough arguments for constructor GpuColumnarBatchSerializer: (metrics: Map[String,com.nvidia.spark.rapids.GpuMetric], dataTypes: Array[org.apache.spark.sql.types.DataType], useKudo: Boolean, kudoMeasureBufferCopy: Boolean)com.nvidia.spark.rapids.GpuColumnarBatchSerializer.
Unspecified value parameter kudoMeasureBufferCopy.[ERROR]     new GpuColumnarBatchSerializer(allMetrics,
@gerashegalov gerashegalov added ? - Needs Triage Need team to review and classify bug Something isn't working labels Feb 6, 2025
@razajafri
Copy link
Collaborator

PR merged without running CI on Databricks

@gerashegalov gerashegalov self-assigned this Feb 7, 2025
@gerashegalov gerashegalov removed the ? - Needs Triage Need team to review and classify label Feb 7, 2025
@gerashegalov gerashegalov mentioned this issue Feb 7, 2025
Merged
@gerashegalov gerashegalov linked a pull request Feb 7, 2025 that will close this issue
Use SHUFFLE_KUDO_SERIALIZER_MEASURE_BUFFER_COPY_ENABLED for [databricks] #12075
Merged
@gerashegalov gerashegalov mentioned this issue Feb 7, 2025
Open
gerashegalov added a commit that referenced this issue Feb 7, 2025
@gerashegalov
Use SHUFFLE_KUDO_SERIALIZER_MEASURE_BUFFER_COPY_ENABLED for [databric…
617aeba 
…ks] (#12075)

Resolves #12074

Signed-off-by: Gera Shegalov <[email protected]>

<!--

Thank you for contributing to RAPIDS Accelerator for Apache Spark!

Here are some guidelines to help the review process go smoothly.

1. Please write a description in this text box of the changes that are
being
   made.

2. Please ensure that you have written units tests for the changes
made/features
   added.

3. If you are closing an issue please use one of the automatic closing
words as
noted here:
https://help.github.com/articles/closing-issues-using-keywords/

4. If your pull request is not ready for review but you want to make use
of the
continuous integration testing facilities please label it with `[WIP]`.

5. If your pull request is ready to be reviewed without requiring
additional
   work on top of it, then remove the `[WIP]` label (if present).

6. Once all work has been done and review has taken place please do not
add
features or make changes out of the scope of those requested by the
reviewer
(doing this just add delays as already reviewed code ends up having to
be
re-reviewed/it is hard to tell what is new etc!). Further, please avoid
rebasing your branch during the review process, as this causes the
context
of any comments made by reviewers to be lost. If conflicts occur during
review then they should be resolved by merging into the branch used for
   making the pull request.

Many thanks in advance for your cooperation!

-->

---------

Signed-off-by: Gera Shegalov <[email protected]>
@gerashegalov
Copy link
Collaborator Author

Fixed by #12075

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerashegalov gerashegalov

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use SHUFFLE_KUDO_SERIALIZER_MEASURE_BUFFER_COPY_ENABLED for [databricks] gerashegalov/spark-rapids
2 participants
@gerashegalov @razajafri