Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle package entry compression #55

Merged
merged 2 commits into from
Aug 17, 2023
Merged

Conversation

javagl
Copy link
Contributor

@javagl javagl commented Aug 8, 2023

There are some details about the compression of the entries of 3D Tiles packages (.3dtiles and .3tz files) that are not yet fully specified, and where the implementation may have to be updated, based on the decisions that are made for the specification.

This PR covers a tiny detail that came out of a real-world example:

A 3TZ file may contain entries that are compressed. And they may be compressed with ZSTD or with DEFLATE. For now, this PR changes the behavior of the 3TZ-based TilesetSource implementation: When it encounters entries that are compressed with DEFLATE, then they will be uncompressed transparently.

Whether or not this should also happen for ZSTD- or GZIP compressed entries is not yet decided.

@javagl
Copy link
Contributor Author

javagl commented Aug 14, 2023

This PR covers two things:

  • Handle the case that entries of .3tz files are compressed with DEFLATE. These entries will be uncompressed transparently, and the corresponding method will always and only return the uncompressed data.
  • Fix some issues for large ZIP files in the "legacy" ArchiveFunctions3tz class. The latest state there did not properly handle certain data layouts for the extras block of ZIP entries (details from the spec added as a comment). Also, some method used the number type where it is safer to use bigint - so now the implementation is using bigint wherever possible.

Some details about the handling of compressed entries in packages are not yet defined. I tried to summarize them in #56 .

@javagl javagl marked this pull request as ready for review August 14, 2023 14:41
@lilleyse lilleyse merged commit b17ddd1 into main Aug 17, 2023
2 checks passed
@lilleyse lilleyse deleted the package-entry-compression branch August 17, 2023 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants