Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ParquetExtractor - Conver Parquet Schema to Flow Schema #933

Closed
norberttech opened this issue Jan 27, 2024 · 1 comment
Closed

ParquetExtractor - Conver Parquet Schema to Flow Schema #933

norberttech opened this issue Jan 27, 2024 · 1 comment
Assignees
@norberttech
Milestone
0.6.0

Comments

@norberttech
Copy link
Member

norberttech commented Jan 27, 2024 •
edited
Loading

As it was already proven in #928 knowing schema upfront can significantly improve turning datasets into Rows. Parquet (but also avro) is format that comes with predefined schema, all we need to do is to read it and convert it into Flow Schema and later pass it into array_to_rows function used in extractors.
@norberttech norberttech converted this from a draft issue Jan 27, 2024
@norberttech norberttech added this to the 0.6.0 milestone Jan 27, 2024
@norberttech norberttech moved this from Todo to On Hold in Roadmap Feb 8, 2024
@norberttech norberttech moved this from On Hold to In Progress in Roadmap Feb 8, 2024
@norberttech norberttech self-assigned this Feb 8, 2024
@norberttech
Copy link
Member Author

Resolved by #975

@github-project-automation github-project-automation bot moved this from In Progress to Done in Roadmap Feb 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@norberttech norberttech

Labels
None yet
Projects
Roadmap
Archived in project
Milestone
0.6.0
Development

No branches or pull requests
1 participant
@norberttech