Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement a streaming merge sorted node #20960

Merged
merged 10 commits into from
Feb 1, 2025

Conversation

coastalwhite
Copy link
Collaborator

No description provided.

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Jan 28, 2025
@coastalwhite coastalwhite force-pushed the feat/streaming-merge-sorted branch from c6049f1 to 52632d1 Compare January 29, 2025 12:14
@coastalwhite coastalwhite marked this pull request as ready for review January 29, 2025 21:59
crates/polars-stream/src/nodes/merge_sorted.rs Outdated Show resolved Hide resolved
crates/polars-stream/src/nodes/merge_sorted.rs Outdated Show resolved Hide resolved
crates/polars-stream/src/nodes/merge_sorted.rs Outdated Show resolved Hide resolved
@coastalwhite coastalwhite force-pushed the feat/streaming-merge-sorted branch from 374cfdf to ea36e6d Compare January 31, 2025 12:01
@@ -156,7 +156,7 @@ impl SeriesTrait for SeriesWrap<TimeChunked> {
}
fn split_at(&self, offset: i64) -> (Series, Series) {
let (a, b) = self.0.split_at(offset);
(a.into_series(), b.into_series())
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drive by: this was producing wrong results for split_at

@@ -16,6 +16,11 @@ pub fn get_ideal_morsel_size() -> usize {
})
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive-by: some docs for MorselSeq

@coastalwhite coastalwhite requested a review from orlp January 31, 2025 12:08
Copy link

codecov bot commented Jan 31, 2025

Codecov Report

Attention: Patch coverage is 66.46154% with 109 lines in your changes missing coverage. Please review.

Project coverage is 79.24%. Comparing base (8837dc8) to head (6396ca5).
Report is 20 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-stream/src/nodes/merge_sorted.rs 66.66% 108 Missing ⚠️
...tes/polars-core/src/series/implementations/time.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #20960      +/-   ##
==========================================
+ Coverage   79.14%   79.24%   +0.10%     
==========================================
  Files        1583     1583              
  Lines      225119   225388     +269     
  Branches     2581     2581              
==========================================
+ Hits       178162   178613     +451     
+ Misses      46367    46185     -182     
  Partials      590      590              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ritchie46 ritchie46 merged commit 82b738b into pola-rs:main Feb 1, 2025
27 checks passed
@coastalwhite coastalwhite deleted the feat/streaming-merge-sorted branch February 1, 2025 15:19
@c-peters c-peters added the accepted Ready for implementation label Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants