Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-45649: [GLib] Add GArrowBinaryViewArray #45650

Merged
merged 4 commits into from
Mar 3, 2025

Conversation

hiroyuki-sato
Copy link
Collaborator

@hiroyuki-sato hiroyuki-sato commented Mar 2, 2025

Rationale for this change

arrow::BinaryViewArray is available in the C++ API.
But, GLib doesn't support that method yet.

What changes are included in this PR?

Add GArrowBinaryViewArray for wrapping arrow::BinaryViewArray class

Are these changes tested?

Yes.

Are there any user-facing changes?

Yes.

Copy link

github-actions bot commented Mar 2, 2025

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

@hiroyuki-sato hiroyuki-sato changed the title [WIP] GH-45649: [GLib] Add GArrowBinaryViewArray GH-45649: [GLib] Add GArrowBinaryViewArray Mar 3, 2025
@hiroyuki-sato hiroyuki-sato marked this pull request as ready for review March 3, 2025 02:39
@hiroyuki-sato hiroyuki-sato requested a review from kou as a code owner March 3, 2025 02:39
Copy link

github-actions bot commented Mar 3, 2025

⚠️ GitHub issue #45649 has been automatically assigned in GitHub to PR creator.

class TestBinaryViewArray < Test::Unit::TestCase
def test_new
view_bytes = 16
buffer_string = "test"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we improve variable name something like the following?

Suggested change
buffer_string = "test"
short_binary_data = "test"


class TestBinaryViewArray < Test::Unit::TestCase
def test_new
view_bytes = 16
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we improve variable name something like the following?

Suggested change
view_bytes = 16
short_string_space = 12

(I think that we don't need to include length space here.)

FYI: The specification: https://arrow.apache.org/docs/format/Columnar.html#variable-size-binary-view-layout

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got the exception Error: test_new(TestBinaryViewArray): Arrow::Error::Invalid: [array][validate-full]: Invalid: Buffer #1 too small in array of type binary_view and length 1: expected at least 16 byte(s), got 8 if I don't use padding.

  def test_new
    # https://arrow.apache.org/docs/format/Columnar.html#variable-size-binary-view-layout
    short_binary_data = "test"
    short_view_buffer_space = 12
    short_view_buffer = [short_binary_data.size].pack("l")
    #short_view_buffer += short_binary_data.ljust(short_view_buffer_space, "\x00")
    short_view_buffer += short_binary_data

    arrow_view_buffer = Arrow::Buffer.new(short_view_buffer)
    arrow_data_buffer = Arrow::Buffer.new(short_binary_data)
    bitmap = Arrow::Buffer.new([0b1].pack("C*"))

    binary_view_array = Arrow::BinaryViewArray.new(1,
                                                   arrow_view_buffer,
                                                   [arrow_data_buffer],
                                                   bitmap,
                                                   0,
                                                   0)
    assert do
      binary_view_array.validate_full
    end
    assert_equal(short_binary_data, binary_view_array.get_value(0).to_s)
  end

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Mar 3, 2025
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Mar 3, 2025
Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Mar 3, 2025
@kou kou merged commit d4d26fe into apache:main Mar 3, 2025
11 checks passed
@kou kou removed the awaiting merge Awaiting merge label Mar 3, 2025
@hiroyuki-sato hiroyuki-sato deleted the topic/garrow-binary-view-array branch March 4, 2025 00:17
Copy link

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit d4d26fe.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 11 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants