fix(iceberg): Date partition value parse issue #12126

nmahadevuni · 2025-01-20T19:22:02Z

Iceberg partition values are already in daysSinceEpoch, but in velox we assume its in date form and try to convert as with Hive. Fixed this.

netlify · 2025-01-20T19:22:31Z

✅ Deploy Preview for meta-velox canceled.

Name	Link
🔨 Latest commit	`b0596f0`
🔍 Latest deploy log	https://app.netlify.com/sites/meta-velox/deploys/67af889cc7f37300089a848a

czentgr · 2025-01-21T14:46:52Z

velox/connectors/hive/HiveConnectorUtil.cpp

-    return applyFilter(*filter, result.value());
+    int32_t result = 0;
+    if (tableFormat == SplitReader::TableFormat::kIceberg) {
+      result = boost::lexical_cast<int32_t>(partitionValue.c_str());


Can we not use std::stoi here instead of including a boost header?
In other comments i see that this function is slow. This could be a problem?

Also we wouldn't need a new include and add a new dependency here.

std::stoi is converting a string like "2022-04-05" to int value 2022, so its not safe and may lead to wrong results.

std::stoi should work here. https://www.geeksforgeeks.org/convert-string-to-int-in-cpp/

std::stoi will work for converting int string to int. But it also doesn't throw error if we input a string like "2022-04-05" as it just converts it to 2022.

Well, 2022-04-05 is parsed by boost into what number? It is not a valid integer in the first place. So either function will not work.

Also you are forgetting that the string is actually the days since epoch like you have in your description. Which means it is not an actual date formatted string. If this was the case you need a date parser here and not a string to int parse function in the first place.

changed to use std::stoi

aditi-pandit

@nmahadevuni : Thanks for this fix. Have bunch of review comments.

velox/connectors/hive/HiveConnectorUtil.cpp

aditi-pandit · 2025-01-22T03:13:01Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+          partitionKeys = {},
+      const std::vector<std::string> filters = {},
+      const std::string duckDbSql = "",
+      const int32_t numPrefetchSplits = 0) {


Why do you have this parameter to the function ? Since it seems like we never test its use really.

Removed this.

aditi-pandit · 2025-01-22T03:15:28Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+  partitionKeys["ds"] = "17627";
+
+  std::vector<RowVectorPtr> dataVectors;
+  VectorPtr c0 = vectorMaker_.flatVector<int64_t>({1});


vectorMaker_ is deprecated. Use makeFlatVector API instead.

aditi-pandit · 2025-01-22T03:18:22Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

@@ -477,6 +514,15 @@ class HiveIcebergTest : public HiveConnectorTestBase {
    return PlanBuilder(pool_.get()).tableScan(rowType_).planNode();
  }

+  core::PlanNodePtr tableScanNode(


Don't think there is a need for this function as its only used in a single place and doesn't really represent anything.

velox/connectors/hive/SplitReader.cpp

aditi-pandit · 2025-01-22T03:31:19Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+      const std::unordered_map<std::string, std::optional<std::string>>
+          partitionKeys = {},
+      const std::vector<std::string> filters = {},
+      const std::string duckDbSql = "",


It is not reasonable to have an empty duckDbSql for this function, as that is the main sql string to verify the plan results with. Since we know the plan we are generating, it might be better to build the duckDBSql in this function itself.

For this test case, we don't know how to generate the duckDbSql, when we add more test cases, we can enhance this function.

I added an empty string check.

In this code a very specific plan with a TableScanNode and partitionfilters, columnfilters is setup. This maps to a quite precise Sql. We could just build it in the logic.

But I'm also fine with the sql passed as a parameter.

We should remove the default "" value for this parameter.

In this test case, we generate int vector for date values and create the data file which is ok for velox, but cannot create duckdb table with the same vectors, so using a sql statement to verify.

aditi-pandit · 2025-01-22T04:37:34Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+    auto scanNodeId = plan->id();
+    auto it = planStats.find(scanNodeId);
+    ASSERT_TRUE(it != planStats.end());
+    ASSERT_TRUE(it->second.peakMemoryBytes > 0);


Why are you testing this ? Isn't matching results sufficient ?

not required, removed it.

aditi-pandit · 2025-01-22T06:49:24Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

@@ -225,6 +226,41 @@ class HiveIcebergTest : public HiveConnectorTestBase {
    ASSERT_TRUE(it->second.peakMemoryBytes > 0);
  }

+  void assertQuery(


Maybe rename this function to assertPartitionKey.

Want to keep this name generic, since it could be used to test any case.

This method is building a very specific plan with TableScanNode and IcebergSplits... The assertQuery function name is being used in all the TestBase classes for very generic usage.

assertPartitionKey also doesn't seem to be the right name, since we can just pass empty partition key. It is just another overloaded assertQuery method.

majetideepak · 2025-01-23T00:58:25Z

velox/connectors/hive/HiveConnectorUtil.cpp

+    if (tableFormat == SplitReader::TableFormat::kIceberg) {
+      result = boost::lexical_cast<int32_t>(partitionValue.c_str());
+    } else {
+      result = DATE()->toDays((folly::StringPiece)partitionValue);


please fix this to use C++ cast since we are touching this code.

Sorry, didn't get it. toDays converts date into daysSinceEpoch, where to use C++ cast?

The C++ style cast:

static_cast<folly::StringPiece>(partitionValue)

See the example in a line you actually removed (below in the review).

Added C++ cast in both places.

majetideepak · 2025-01-23T00:59:49Z

velox/connectors/hive/SplitReader.h

@@ -154,6 +157,7 @@ class SplitReader {
      std::shared_ptr<HiveColumnHandle>>* const partitionKeys_;
  const ConnectorQueryCtx* connectorQueryCtx_;
  const std::shared_ptr<const HiveConfig> hiveConfig_;
+  const TableFormat tableFormat_;


don't need a const for enums and scalars.

I made it a const reference.

Are we missing something? This is just a plain const and not a const reference. Also for enums you don't need a reference. And the argument to the constructor is not a const & either.

Sorry, I was looking at a different place altogether. You are right. I removed all const references.

aditi-pandit · 2025-01-23T05:35:41Z

velox/connectors/hive/HiveConnectorUtil.cpp

@@ -634,12 +636,16 @@ namespace {
 bool applyPartitionFilter(
    const TypePtr& type,
    const std::string& partitionValue,
-    common::Filter* filter) {
+    common::Filter* filter,
+    const SplitReader::TableFormat& tableFormat) {


Its better to move the read-only const& parameters before the writable * ones. So lets move tableFormat to the first parameter.

velox/connectors/hive/SplitReader.cpp

aditi-pandit · 2025-01-23T05:51:47Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+      const std::unordered_map<std::string, std::optional<std::string>>
+          partitionKeys = {},
+      const std::vector<std::string> filters = {},
+      const std::string duckDbSql = "",


In this code a very specific plan with a TableScanNode and partitionfilters, columnfilters is setup. This maps to a quite precise Sql. We could just build it in the logic.

But I'm also fine with the sql passed as a parameter.

aditi-pandit · 2025-01-23T05:52:19Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+      const std::unordered_map<std::string, std::optional<std::string>>
+          partitionKeys = {},
+      const std::vector<std::string> filters = {},
+      const std::string duckDbSql = "",


We should remove the default "" value for this parameter.

nmahadevuni · 2025-01-24T06:55:45Z

Thank you @aditi-pandit @majetideepak @czentgr . I have addressed your comments. Please review.

nmahadevuni · 2025-01-27T17:26:24Z

@majetideepak @aditi-pandit @czentgr Addressed your comments. Please have a look.

velox/connectors/hive/HiveConnectorUtil.cpp

velox/connectors/hive/SplitReader.cpp

nmahadevuni · 2025-01-28T06:50:07Z

The date values are set in daysSinceEpoch in the FileScanTask returned by the Iceberg API. If we want to change that, that conversion back to string format has to happen in Coordinator. But Presto Java workers do not have any issue with that, and will need to be changed if we change it back to date string. So instead of changing at two places in Java, I fixed it in Velox. testFilters and applyPartitionFilter are static functions, setPartitionValue is the only class member. We will override and do special handling in IcebergSplitReader::setPartitionValue? @majetideepak

majetideepak · 2025-01-28T15:02:18Z

The date values are set in daysSinceEpoch in the FileScanTask returned by the Iceberg API.

@nmahadevuni Let's add bool isPartitionValueDaysSinceEpoch_ to the ScanSpec.

nmahadevuni · 2025-01-29T04:00:47Z

The date values are set in daysSinceEpoch in the FileScanTask returned by the Iceberg API.

@nmahadevuni Let's add bool isPartitionValueDaysSinceEpoch_ to the ScanSpec.

As discussed on slack, added the bool to SplitReader class. Please review @majetideepak

velox/connectors/hive/HiveConnectorUtil.cpp

velox/connectors/hive/HiveConnectorUtil.h

velox/exec/tests/utils/PlanBuilder.h

majetideepak · 2025-01-29T18:25:12Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+  VectorPtr ds = makeFlatVector<int32_t>((std::vector<int32_t>){17627});
+  dataVectors.push_back(makeRowVector({"c0", "ds"}, {c0, ds}));
+
+  assertQuery(


writeToFile can be moved here since it is the same data.
I feel it is cleaner to create the plan and call HiveConnectorTestBase::assertQuery here.
This is inline with all the partition tests inside TableScanTest.cpp

Agree... The current assertQuery function can be a lambda here for reuse.

+1. Inlining the partition tests here fits better with the rest of the tests in this file like assertPositionalDeletes.

majetideepak · 2025-01-29T18:26:00Z

@Yuhta can you take a look at this change? Thanks.

velox/connectors/hive/HiveConnectorUtil.cpp

majetideepak · 2025-02-05T17:38:11Z

velox/connectors/hive/TableHandle.h

@@ -45,12 +47,14 @@ class HiveColumnHandle : public ColumnHandle {
      ColumnType columnType,
      TypePtr dataType,
      TypePtr hiveType,
-      std::vector<common::Subfield> requiredSubfields = {})
+      std::vector<common::Subfield> requiredSubfields = {},
+      ValueType valueType = ValueType::kDefault)


I think we should follow the HiveTableHandle and add a generic argument like
const std::unordered_map<std::string, std::string>& columnParameters = {}

We can then define struct ColumnParameter similar to struct TableParameter and add an entry for our need. Example: key=partition.date.value.format, value=daysSinceEpoch/ISODateFormat as from our discussion offline.

@Yuhta Do you agree with this API approach?

nmahadevuni · 2025-02-07T17:57:45Z

@majetideepak @Yuhta Thanks for the review. Made the changes as suggested. Please have a look.

majetideepak

@nmahadevuni the Velox API looks good to me. Found some more comments.

velox/connectors/hive/HiveConnectorUtil.cpp

majetideepak · 2025-02-10T17:49:33Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+      partitionColumns.insert(partitionKey.first);
+    }
+
+    //    auto plan =


nit: remove commented code.

majetideepak · 2025-02-10T17:52:21Z

@Yuhta can you comment on this API? Thanks.

aditi-pandit

Thanks @nmahadevuni. Have few questions.

aditi-pandit · 2025-02-11T00:26:57Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+  assertQuery(
+      rowType, dataVectors, "SELECT 1, '2018-04-06'", partitionKeys, {});
+
+  std::vector<std::string> filters = {"ds = date'2018-04-06'"};


This test is very simple. The partition keys, filters are the same and there is only one matching row that is selected. Would be good to add rows in the input that don't match the partition key and are filtered out. Also would be good to add a filter for another non-partition key value that matches an input row.

aditi-pandit · 2025-02-11T00:30:44Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+  VectorPtr ds = makeFlatVector<int32_t>((std::vector<int32_t>){17627});
+  dataVectors.push_back(makeRowVector({"c0", "ds"}, {c0, ds}));
+
+  assertQuery(


+1. Inlining the partition tests here fits better with the rest of the tests in this file like assertPositionalDeletes.

aditi-pandit · 2025-02-11T00:34:28Z

velox/connectors/hive/HiveConnectorUtil.cpp

+    int32_t result = 0;
+    // Iceberg partition values are already in daysSinceEpoch, no need to
+    // convert.
+    if (columnParameters.count(


This check is repeated twice. Can we abstract this check as a method of ColumnHandle ?

aditi-pandit · 2025-02-11T00:38:25Z

velox/connectors/hive/TableHandle.h

+  const std::unordered_map<std::string, std::string>& columnParameters() const {
+    return columnParameters_;
+  }
+
  std::string toString() const;

  folly::dynamic serialize() const override;


The serialize and creata methods should be enhanced to also serialize/deserialize the columnParameters_ in the HiveColumnHandle.

Do we need to change serde and toString methods? The columnParameters_ will be set in Prestissimo code. Even tableParameters_ from HiveTableHandle class is implemented the same way.

@nmahadevuni : Yes, serde and toString methods should always be updated when we update classes that appear in a Velox plan.

toString is used in printPlanWithStats debugging utilities https://facebookincubator.github.io/velox/develop/debugging/print-plan-with-stats.html and the serde methods have been used in non-Presto use-cases.

tableParameters_ should be handled this way as well. Its a bug if it is not.

aditi-pandit · 2025-02-11T00:38:59Z

velox/connectors/hive/TableHandle.h

+  const std::unordered_map<std::string, std::string>& columnParameters() const {
+    return columnParameters_;
+  }
+
  std::string toString() const;


toString() should also be enhanced to show the columnParameters_.

Please update unit tests as well.

aditi-pandit · 2025-02-11T00:43:38Z

velox/connectors/hive/TableHandle.h

@@ -115,6 +121,7 @@ class HiveColumnHandle : public ColumnHandle {
  const TypePtr dataType_;
  const TypePtr hiveType_;
  const std::vector<common::Subfield> requiredSubfields_;
+  const std::unordered_map<std::string, std::string> columnParameters_;


Please add a comment here indicating that these are used for metadata like column date value format.

If you have a link to how these are populated that will be great as well.

This change would be accompanied by presto-native code to populate these fields in IcebergPrestoToVeloxConnector::toVeloxColumnHandle method. So is there a change in the protocol classes as well to pass these from the Iceberg connector ? Do you the e2e Prestissimo PR ? Would be great to take a look at that to understand the scope of this change.

Yes. We will need to change IcebergPrestoToVeloxConnector::toVeloxColumnHandle. This change will need to be merged first for me to work on the Prestissimo PR.

Would be great to see the Prestissimo code at the same time to know this works e2e.

You can change the Velox submodule in the Presto project to pick up the changes from your branch https://git-scm.com/book/en/v2/Git-Tools-Submodules section "Pulling in Upstream Changes from the Submodule Remote"
and try out this code with presto-native-execution test.

See prestodb/presto#24138 for how a PR looks with Velox submodule changes for a branch with local commits.

nmahadevuni · 2025-02-11T16:40:03Z

@majetideepak @aditi-pandit Thank you. I have addressed your comments. Please review.

Yuhta · 2025-02-12T14:58:33Z

velox/connectors/hive/TableHandle.h

@@ -115,6 +129,9 @@ class HiveColumnHandle : public ColumnHandle {
  const TypePtr dataType_;
  const TypePtr hiveType_;
  const std::vector<common::Subfield> requiredSubfields_;
+  // The column parameters are used for metadata like Iceberg date partition
+  // value format.
+  const std::unordered_map<std::string, std::string> columnParameters_;


Let's just make it typed struct as there is no need for serialization for this parameter. Something like this:

struct ColumnParseParameters { enum PartitionDateValueFormat { kISO8601, kDaysSinceEpoch, } partitionDateValueFormat; };

Thanks @Yuhta . I have made the changes. Also, the draft changes on Prestissimo side are at nmahadevuni/presto@32b22cb. Please review @majetideepak @aditi-pandit

velox/exec/tests/utils/PlanBuilder.h

velox/exec/tests/utils/PlanBuilder.cpp

nmahadevuni · 2025-02-13T12:21:40Z

Have made the changes. Please review. @Yuhta @majetideepak @aditi-pandit

majetideepak

@nmahadevuni this looks nice! Few comments.
@aditi-pandit can you take another look?

velox/connectors/hive/TableHandle.h

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

majetideepak

Thanks, @nmahadevuni
I will add the merge label once Aditi approves.

aditi-pandit

Thanks @nmahadevuni

facebook-github-bot · 2025-02-14T02:19:01Z

@pedroerp has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

pedroerp · 2025-02-14T16:27:39Z

@nmahadevuni looks like this unit test failed internally. Could you take a look?

'velox/connectors/hive/iceberg/tests:tests - HiveIcebergTest.testPartitionedRead'

pedroerp · 2025-02-14T16:28:24Z

it's a memory leak detected by ASAN:

=================================================================
==8199==ERROR: LeakSanitizer: detected memory leaks

Indirect leak of 272 byte(s) in 1 object(s) allocated from:
    #0 0x881d8d in operator new(unsigned long) (/data/sandcastle/boxes/eden-trunk-hg-full-fbsource/buck-out/v2/gen/fbcode/07838712e61ba1ef/velox/connectors/hive/iceberg/tests/__tests__/tests+0x881d8d)
    #1 0x5739a5 in facebook::velox::exec::test::PlanBuilder::startTableScan() fbcode/velox/exec/tests/utils/PlanBuilder.h:305
    #2 0x3708b1 in facebook::velox::connector::hive::iceberg::HiveIcebergTest_testPartitionedRead_Test::TestBody() fbcode/velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp:741
    #3 0x7f62dcd2b1be in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) fbsource/src/gtest.cc:2675
    #4 0x7f62dcd2aa44 in testing::Test::Run() fbsource/src/gtest.cc:2692
    #5 0x7f62dcd3068f in testing::TestInfo::Run() fbsource/src/gtest.cc:2841
    #6 0x7f62dcd38646 in testing::TestSuite::Run() fbsource/src/gtest.cc:3020
    #7 0x7f62dcd73fab in testing::internal::UnitTestImpl::RunAllTests() fbsource/src/gtest.cc:5925
    #8 0x7f62dcd7300b in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) fbsource/src/gtest.cc:2675
    #9 0x7f62dcd72549 in testing::UnitTest::Run() fbsource/src/gtest.cc:5489
    #10 0x7f62dcc06980 in RUN_ALL_TESTS() fbsource/gtest/gtest.h:2317
    #11 0x7f62dcc0660f in main fbcode/common/gtest/LightMain.cpp:20
    #12 0x7f62d202c656 in __libc_start_call_main /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #13 0x7f62d202c717 in [__libc_start_main@GLIBC_2.2.5](mailto:__libc_start_main@GLIBC_2.2.5) /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../csu/libc-start.c:409:3
    #14 0x360130 in _start /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../sysdeps/x86_64/start.S:116

Indirect leak of 256 byte(s) in 2 object(s) allocated from:
    #0 0x881d8d in operator new(unsigned long) (/data/sandcastle/boxes/eden-trunk-hg-full-fbsource/buck-out/v2/gen/fbcode/07838712e61ba1ef/velox/connectors/hive/iceberg/tests/__tests__/tests+0x881d8d)
    #1 0x7f62d624efe9 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<facebook::velox::connector::hive::HiveColumnHandle, std::allocator<facebook::velox::connector::hive::HiveColumnHandle>, (__gnu_cxx::_Lock_policy)2>>::allocate(unsigned long, void const*) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/ext/new_allocator.h:128
    #2 0x7f62d624eea0 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::connector::hive::HiveColumnHandle, std::allocator<facebook::velox::connector::hive::HiveColumnHandle>, (__gnu_cxx::_Lock_policy)2>>>::allocate(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::connector::hive::HiveColumnHandle, std::allocator<facebook::velox::connector::hive::HiveColumnHandle>, (__gnu_cxx::_Lock_policy)2>>&, unsigned long) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/allocator.h:197
    #3 0x7f62d624ea40 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::connector::hive::HiveColumnHandle, std::allocator<facebook::velox::connector::hive::HiveColumnHandle>, (__gnu_cxx::_Lock_policy)2>>> std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::connector::hive::HiveColumnHandle, std::allocator<facebook::velox::connector::hive::HiveColumnHandle>, (__gnu_cxx::_Lock_policy)2>>>(std::allocator<std::_Sp_counted_ptr_inplace<facebook::velox::connector::hive::HiveColumnHandle, std::allocator<facebook::velox::connector::hive::HiveColumnHandle>, (__gnu_cxx::_Lock_policy)2>>&) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/allocated_ptr.h:97
    #4 0x7f62d62d91d8 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<facebook::velox::connector::hive::HiveColumnHandle, std::allocator<facebook::velox::connector::hive::HiveColumnHandle>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>&, facebook::velox::connector::hive::HiveColumnHandle::ColumnType, std::shared_ptr<facebook::velox::Type const> const&, std::shared_ptr<facebook::velox::Type const> const&>(facebook::velox::connector::hive::HiveColumnHandle*&, std::_Sp_alloc_shared_tag<std::allocator<facebook::velox::connector::hive::HiveColumnHandle>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>&, facebook::velox::connector::hive::HiveColumnHandle::ColumnType&&, std::shared_ptr<facebook::velox::Type const> const&, std::shared_ptr<facebook::velox::Type con
...
...

nmahadevuni · 2025-02-14T17:02:50Z

@nmahadevuni looks like this unit test failed internally. Could you take a look?

'velox/connectors/hive/iceberg/tests:tests - HiveIcebergTest.testPartitionedRead'

Thank you. Will have a look.

majetideepak · 2025-02-14T18:00:32Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+           std::move(requiredSubFields),
+           columnParseParameters)});
+
+  auto planBuilder = new PlanBuilder(pool_.get());


This is a leak.
You can use auto plan = PlanBuilder(pool_.get()).startTableScan()....

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

majetideepak · 2025-02-14T18:02:11Z

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

+
+  // Test filter on non-partitioned non-date column
+  std::vector<std::string> nonPartitionFilters = {"c0 = 1"};
+  plan = planBuilder->startTableScan()


PlanBuilder().startTableScan()

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

nmahadevuni · 2025-02-14T18:18:08Z

Fixed it and ran the leaks tool on Mac to check. @pedroerp @majetideepak

Process:         velox_hive_iceberg_test [30630]
Path:            /Users/USER/*/velox_hive_iceberg_test
Load Address:    0x1007a4000
Identifier:      velox_hive_iceberg_test
Version:         0
Code Type:       ARM64
Platform:        macOS
Parent Process:  leaks [30629]

Date/Time:       2025-02-14 23:40:39.413 +0530
Launch Time:     2025-02-14 23:40:38.511 +0530
OS Version:      macOS 14.5 (23F79)
Report Version:  7
Analysis Tool:   /usr/bin/leaks

Physical footprint:         40.0M
Physical footprint (peak):  40.8M
Idle exit:                  untracked
----

leaks Report Version: 4.0, multi-line stacks
Process 30630: 1001 nodes malloced for 238 KB
Process 30630: 0 leaks for 0 total leaked bytes.

facebook-github-bot · 2025-02-14T18:30:37Z

@pedroerp has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

nmahadevuni requested a review from majetideepak as a code owner January 20, 2025 19:22

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 20, 2025

nmahadevuni requested review from aditi-pandit and czentgr January 21, 2025 10:26

czentgr reviewed Jan 21, 2025

View reviewed changes

aditi-pandit reviewed Jan 22, 2025

View reviewed changes

majetideepak reviewed Jan 23, 2025

View reviewed changes

aditi-pandit reviewed Jan 23, 2025

View reviewed changes

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch 2 times, most recently from d3564ed to c7843f4 Compare January 24, 2025 06:55

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from c7843f4 to 67b025c Compare January 27, 2025 06:58

majetideepak reviewed Jan 27, 2025

View reviewed changes

velox/connectors/hive/HiveConnectorUtil.cpp Outdated Show resolved Hide resolved

velox/connectors/hive/SplitReader.cpp Show resolved Hide resolved

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from 67b025c to ff840fc Compare January 29, 2025 03:58

majetideepak reviewed Jan 29, 2025

View reviewed changes

Yuhta reviewed Jan 29, 2025

View reviewed changes

velox/connectors/hive/HiveConnectorUtil.cpp Outdated Show resolved Hide resolved

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from ff840fc to 2f5fa0a Compare February 5, 2025 17:18

majetideepak reviewed Feb 5, 2025

View reviewed changes

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from 2f5fa0a to 30f5dad Compare February 7, 2025 17:56

majetideepak reviewed Feb 10, 2025

View reviewed changes

aditi-pandit reviewed Feb 11, 2025

View reviewed changes

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from 30f5dad to 9e5d6d2 Compare February 11, 2025 16:39

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch 2 times, most recently from a98d364 to 3804e10 Compare February 12, 2025 11:53

Yuhta reviewed Feb 12, 2025

View reviewed changes

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from 3804e10 to f6964f1 Compare February 12, 2025 16:55

Yuhta approved these changes Feb 12, 2025

View reviewed changes

velox/exec/tests/utils/PlanBuilder.h Outdated Show resolved Hide resolved

majetideepak reviewed Feb 12, 2025

View reviewed changes

velox/exec/tests/utils/PlanBuilder.cpp Outdated Show resolved Hide resolved

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from f6964f1 to 27baa3c Compare February 13, 2025 08:21

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from 27baa3c to 52b81fa Compare February 13, 2025 13:26

majetideepak reviewed Feb 13, 2025

View reviewed changes

velox/connectors/hive/TableHandle.h Outdated Show resolved Hide resolved

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp Show resolved Hide resolved

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp Outdated Show resolved Hide resolved

majetideepak approved these changes Feb 13, 2025

View reviewed changes

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from 52b81fa to 457e82a Compare February 13, 2025 17:44

aditi-pandit approved these changes Feb 13, 2025

View reviewed changes

majetideepak added the ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall label Feb 13, 2025

majetideepak reviewed Feb 14, 2025

View reviewed changes

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp Show resolved Hide resolved

majetideepak reviewed Feb 14, 2025

View reviewed changes

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp Show resolved Hide resolved

majetideepak reviewed Feb 14, 2025

View reviewed changes

velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp Show resolved Hide resolved

fix(iceberg): Date partition value parse issue

b0596f0

nmahadevuni force-pushed the fix_ice_date_partition_value_parse branch from 457e82a to b0596f0 Compare February 14, 2025 18:16

fix(iceberg): Date partition value parse issue #12126

Are you sure you want to change the base?

fix(iceberg): Date partition value parse issue #12126

Conversation

nmahadevuni commented Jan 20, 2025 • edited Loading

netlify bot commented Jan 20, 2025 • edited Loading

✅ Deploy Preview for meta-velox canceled.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aditi-pandit left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nmahadevuni commented Jan 24, 2025

nmahadevuni commented Jan 27, 2025

nmahadevuni commented Jan 28, 2025 • edited Loading

majetideepak commented Jan 28, 2025

nmahadevuni commented Jan 29, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

majetideepak commented Jan 29, 2025

majetideepak Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nmahadevuni commented Feb 7, 2025

majetideepak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

majetideepak commented Feb 10, 2025

aditi-pandit left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nmahadevuni commented Feb 11, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nmahadevuni commented Feb 13, 2025

majetideepak left a comment

Choose a reason for hiding this comment

majetideepak left a comment

Choose a reason for hiding this comment

aditi-pandit left a comment

nmahadevuni commented Jan 20, 2025 •

edited

Loading

netlify bot commented Jan 20, 2025 •

edited

Loading

nmahadevuni commented Jan 28, 2025 •

edited

Loading

majetideepak Feb 5, 2025 •

edited

Loading

majetideepak Feb 14, 2025 •

edited

Loading