Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ARCTIC-1668] Upgrade the Iceberg dependency version to 1.3.0 #1725

Merged
merged 26 commits into from
Aug 8, 2023

Conversation

wangtaohz
Copy link
Contributor

Why are the changes needed?

resolve #1668

Brief change log

  • Upgrade the Iceberg dependency version to 1.3.0
  • Adapt to Iceberg API changes

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduces a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@github-actions github-actions bot added module:core Core module module:mixed-hive Hive moduel for Mixed Format type:build labels Jul 20, 2023
@codecov
Copy link

codecov bot commented Jul 20, 2023

Codecov Report

Patch coverage: 61.53% and project coverage change: -0.04% ⚠️

Comparison is base (05d6fbf) 50.98% compared to head (bb2bc83) 50.95%.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1725      +/-   ##
============================================
- Coverage     50.98%   50.95%   -0.04%     
  Complexity     3771     3771              
============================================
  Files           467      467              
  Lines         25376    25379       +3     
  Branches       2581     2581              
============================================
- Hits          12938    12931       -7     
- Misses        11277    11291      +14     
+ Partials       1161     1157       -4     
Flag Coverage Δ
core 51.42% <61.53%> (-0.02%) ⬇️
trino 48.67% <ø> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
.../main/java/com/netease/arctic/op/ArcticUpdate.java 63.00% <0.00%> (-1.29%) ⬇️
.../netease/arctic/hive/op/ReplaceHivePartitions.java 62.91% <0.00%> (-0.85%) ⬇️
...java/org/apache/iceberg/ArcticChangeTableScan.java 79.45% <85.71%> (+1.07%) ⬆️
...java/com/netease/arctic/table/BasicKeyedTable.java 71.18% <100.00%> (ø)
.../com/netease/arctic/hive/table/KeyedHiveTable.java 81.81% <100.00%> (ø)

... and 6 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions github-actions bot added the module:mixed-flink Flink moduel for Mixed Format label Jul 20, 2023
@wangtaohz wangtaohz requested a review from baiyangtx July 20, 2023 13:17
@github-actions github-actions bot added module:mixed-spark Spark module for Mixed Format module:ams-server Ams server module module:mixed-trino trino module for Mixed Format module:ams-dashboard Ams dashboard module labels Jul 21, 2023
@wangtaohz wangtaohz changed the title [ARCTIC-1668] Upgrade the Iceberg dependency version to 1.3.0 [WIP][ARCTIC-1668] Upgrade the Iceberg dependency version to 1.3.0 Jul 24, 2023
@wangtaohz wangtaohz changed the title [WIP][ARCTIC-1668] Upgrade the Iceberg dependency version to 1.3.0 [ARCTIC-1668] Upgrade the Iceberg dependency version to 1.3.0 Jul 25, 2023
ams/api/pom.xml Outdated Show resolved Hide resolved
@github-actions github-actions bot removed module:mixed-spark Spark module for Mixed Format module:ams-server Ams server module module:mixed-trino trino module for Mixed Format module:ams-dashboard Ams dashboard module labels Aug 8, 2023
pom.xml Outdated Show resolved Hide resolved
Copy link
Contributor

@YesOrNo828 YesOrNo828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks good to me. After confirming the prefix typo, I think we could merge it into the master.

Copy link
Contributor

@baiyangtx baiyangtx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhoujinsong zhoujinsong merged commit 4ccd5d7 into apache:master Aug 8, 2023
4 checks passed
ShawHee pushed a commit to ShawHee/arctic that referenced this pull request Dec 29, 2023
…#1725)

* upgrade iceberg to 1.3.0

* fix flink

* fix flink

* remove useless

* change version from 0.5.0-SNAPSHOT to 0.5.1-SNAPSHOT

* update iceberg version 1.3.x in Flink 1.12 module

* update iceberg version 1.3.x in Flink 1.12 module

* ArcticUpdate support toBranch

* fix ci error

* update iceberg version 1.3.x in Flink 1.14 module

* update iceberg version 1.3.x in Flink 1.15 module

* calculate available core

* change max input size per thread from 5GB to 500MB

* change version from 0.5.1-SNAPSHOT to 0.5.0-SNAPSHOT

* fix the hive.version

---------

Co-authored-by: lklhdu <[email protected]>
Co-authored-by: ZhouJinsong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module:core Core module module:mixed-flink Flink moduel for Mixed Format module:mixed-hive Hive moduel for Mixed Format type:build
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Subtask]: Upgrade Iceberg version to 1.3.0
5 participants