Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support metadata tables #823

Open
xxchan opened this issue Dec 18, 2024 · 1 comment
Open

feat: support metadata tables #823

xxchan opened this issue Dec 18, 2024 · 1 comment

Comments

@xxchan
Copy link
Contributor

xxchan commented Dec 18, 2024

In Iceberg Spark/Flink there are metadata tables that provide information around the table: https://iceberg.apache.org/docs/latest/spark-queries/#inspecting-tables

Supporting this in iceberg-rust allows other engines (like RisingWave) to support these "standard" metadata tables.

reference implementation:
- https://py.iceberg.apache.org/api/#inspecting-tables
- https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/inspect.py#L58-L90

List of all metadata tables: https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetadataTableType.java#L23

  • ENTRIES
  • FILES
  • DATA_FILES
  • DELETE_FILES
  • HISTORY
  • METADATA_LOG_ENTRIES
  • SNAPSHOTS feat: support metadata table "snapshots" #822
  • REFS
  • MANIFESTS
  • PARTITIONS
  • ALL_DATA_FILES
  • ALL_DELETE_FILES
  • ALL_FILES
  • ALL_MANIFESTS
  • ALL_ENTRIES
  • POSITION_DELETES
@Fokko
Copy link
Contributor

Fokko commented Dec 18, 2024

Thanks for working on this @xxchan. Metadata tables are very important when it comes to the metadata operations. For example, the snapshots table in #822 will make it easy to perform operations like expiring snapshots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants