-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement lazyframe profiling and optimization toggles #323
Conversation
we are missing a unit test. py-polars do not always unit test this either. We need to extract the curret settings from a LazyFrame / logicalplan. I have previous found bugs in rust-polars by testing such optimization settings. |
@Sicheng-Pan I'm ok with merging this PR now. I think you mentioned you wanted to allow both the setting opt-toggles via collect as in py-polars, but that it also makes sense to se individually as in rust-polars. I'm agree with that, if you want to add that back in. we can just postpone the unit tests until #324 has been merged |
Merge branch 'main' into lazy_profile # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.
This PR is needed for #343 due to opt-toogles + polars CSE feature + robj_to!(i32, robj) @Sicheng-Pan Refactored and added two internal functions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could only review the R part, need someone else for the Rust part
Co-authored-by: Etienne Bacher <[email protected]>
Co-authored-by: Etienne Bacher <[email protected]>
Co-authored-by: Etienne Bacher <[email protected]>
make fmt
Co-authored-by: Etienne Bacher <[email protected]>
@etiennebacher @eitsupi I think this PR is ready to go. It is needed for the Sink PR #343 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just did minor doc changes. @sorhawell since you and @Sicheng-Pan have worked on the Rust part I guess it's already reviewed.
I'm just waiting the CI to pass before merging, thank you both!
#261