Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clarify logic in nth_value window function #14104

Merged
merged 1 commit into from
Jan 14, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 17 additions & 15 deletions datafusion/functions-window/src/nth_value.rs
Original file line number Diff line number Diff line change
Expand Up @@ -345,24 +345,26 @@ impl PartitionEvaluator for NthValueEvaluator {
return ScalarValue::try_from(arr.data_type());
}

// Extract valid indices if ignoring nulls.
// If null values exist and need to be ignored, extract the valid indices.
let valid_indices = if self.ignore_nulls {
// Calculate valid indices, inside the window frame boundaries
// Calculate valid indices, inside the window frame boundaries.
let slice = arr.slice(range.start, n_range);
let valid_indices = slice
.nulls()
.map(|nulls| {
nulls
match slice.nulls() {
Some(nulls) => {
let valid_indices = nulls
.valid_indices()
// Add offset `range.start` to valid indices, to point correct index in the original arr.
.map(|idx| idx + range.start)
.collect::<Vec<_>>()
})
.unwrap_or_default();
if valid_indices.is_empty() {
None
} else {
Some(valid_indices)
.map(|idx| {
// Add offset `range.start` to valid indices, to point correct index in the original arr.
idx + range.start
})
.collect::<Vec<_>>();
if valid_indices.is_empty() {
// If all values are null, return directly.
return ScalarValue::try_from(arr.data_type());
}
Some(valid_indices)
}
None => None,
}
} else {
None
Expand Down
Loading