Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make iceberg table target split size configurable #24419

Open
ZacBlanco opened this issue Jan 23, 2025 · 1 comment · May be fixed by #24417
Open

Make iceberg table target split size configurable #24419

ZacBlanco opened this issue Jan 23, 2025 · 1 comment · May be fixed by #24417

Comments

@ZacBlanco
Copy link
Contributor

ZacBlanco commented Jan 23, 2025

We should have a way to set a table's target split size for the Iceberg connector in Presto. The current default is 128MB and in some cases generates too much overhead. We should be able to set the target split size in order to tune the performance of some or all queries.

Expected Behavior or Use Case

We should be able to set the target split size either via table properties or session properties.

Presto Component, Service, or Connector

Iceberg connector.

Possible Implementation

  • Add session property to configure split size for all queries in a session (overrides table property)
  • Add support for target split size in ALTER TABLE ... SET PROPERTIES statement

Context

During table scan planning within the Iceberg connector, we generate splits based on the iceberg table's read.split.target-size property. For while there was no easy way to set table properties. However #21495 introduced this feature, but still doesn't allow setting arbitrary properties unless we update the set of supported table properties in IcebergAbstractMetadata#setTableProperties.

We should introduce support for setting the table property size to allow tuning this parameter on Iceberg tables.

@ZacBlanco
Copy link
Contributor Author

This implemented in #24417

@ZacBlanco ZacBlanco linked a pull request Jan 28, 2025 that will close this issue
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant