Replies: 1 comment
-
Hi @Mydurian, ParquetSharp doesn't depend on the names of schema nodes matching the specification, following the backward-compatibility rules, but we currently only support lists that are represented with the three level structure and don't handle data that is missing the inner element layer. I don't think there's a simple way to work around this, but we'd be open to accepting a pull request to add backwards-compatibility support for reading files that don't use the standard schema structure. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi Team,
![image](https://private-user-images.githubusercontent.com/48521631/311628622-6132d709-a399-47a7-b825-4203ca47ab1e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk2NzQ4MzgsIm5iZiI6MTczOTY3NDUzOCwicGF0aCI6Ii80ODUyMTYzMS8zMTE2Mjg2MjItNjEzMmQ3MDktYTM5OS00N2E3LWI4MjUtNDIwM2NhNDdhYjFlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjE2VDAyNTUzOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTYwMGZhZjM4OWQ5ZTI0YmQzOTdlZjcyOGJkZDRkZTc0NDEyYjk5ZWUxMjVkNjBiM2QyYTE0NTczMjQ0OGJmMjQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.jflqxL0GyDvN0eGXRwd5GcM0AiV2NHU_xijUJOOXd80)
When referring the Apache official doc for complex logical type.. I see that the structure definition is somewhere different. Could you help elaborate a little bit more on how to handle some backward-compatibility in ParquetSharp? Thanks in advance.
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md
Beta Was this translation helpful? Give feedback.
All reactions