-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-45649: [GLib] Add GArrowBinaryViewArray #45650
Conversation
Thanks for opening a pull request! If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename the pull request title in the following format?
or
See also: |
|
class TestBinaryViewArray < Test::Unit::TestCase | ||
def test_new | ||
view_bytes = 16 | ||
buffer_string = "test" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we improve variable name something like the following?
buffer_string = "test" | |
short_binary_data = "test" |
|
||
class TestBinaryViewArray < Test::Unit::TestCase | ||
def test_new | ||
view_bytes = 16 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we improve variable name something like the following?
view_bytes = 16 | |
short_string_space = 12 |
(I think that we don't need to include length space here.)
FYI: The specification: https://arrow.apache.org/docs/format/Columnar.html#variable-size-binary-view-layout
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got the exception Error: test_new(TestBinaryViewArray): Arrow::Error::Invalid: [array][validate-full]: Invalid: Buffer #1 too small in array of type binary_view and length 1: expected at least 16 byte(s), got 8
if I don't use padding.
def test_new
# https://arrow.apache.org/docs/format/Columnar.html#variable-size-binary-view-layout
short_binary_data = "test"
short_view_buffer_space = 12
short_view_buffer = [short_binary_data.size].pack("l")
#short_view_buffer += short_binary_data.ljust(short_view_buffer_space, "\x00")
short_view_buffer += short_binary_data
arrow_view_buffer = Arrow::Buffer.new(short_view_buffer)
arrow_data_buffer = Arrow::Buffer.new(short_binary_data)
bitmap = Arrow::Buffer.new([0b1].pack("C*"))
binary_view_array = Arrow::BinaryViewArray.new(1,
arrow_view_buffer,
[arrow_data_buffer],
bitmap,
0,
0)
assert do
binary_view_array.validate_full
end
assert_equal(short_binary_data, binary_view_array.get_value(0).to_s)
end
Co-authored-by: Sutou Kouhei <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit d4d26fe. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 11 possible false positives for unstable benchmarks that are known to sometimes produce them. |
Rationale for this change
arrow::BinaryViewArray is available in the C++ API.
But, GLib doesn't support that method yet.
What changes are included in this PR?
Add
GArrowBinaryViewArray
for wrappingarrow::BinaryViewArray
classAre these changes tested?
Yes.
Are there any user-facing changes?
Yes.