spark-sql-Expression-ImperativeAggregate.adoc

ImperativeAggregate — Contract for Aggregate Function Expressions with Imperative Methods

ImperativeAggregate is the contract for aggregate functions that are expressed in terms of imperative initialize, update, and merge methods (that operate on Row-based aggregation buffers).

ImperativeAggregate is a Catalyst expression with CodegenFallback.

Table 1. ImperativeAggregate’s Direct Implementations

Name	Description
HyperLogLogPlusPlus
PivotFirst
ScalaUDAF
TypedImperativeAggregate

ImperativeAggregate Contract

package org.apache.spark.sql.catalyst.expressions.aggregate

abstract class ImperativeAggregate {
  def initialize(mutableAggBuffer: InternalRow): Unit
  val inputAggBufferOffset: Int
  def merge(mutableAggBuffer: InternalRow, inputAggBuffer: InternalRow): Unit
  val mutableAggBufferOffset: Int
  def update(mutableAggBuffer: InternalRow, inputRow: InternalRow): Unit
  def withNewInputAggBufferOffset(newInputAggBufferOffset: Int): ImperativeAggregate
  def withNewMutableAggBufferOffset(newMutableAggBufferOffset: Int): ImperativeAggregate
}

Table 2. ImperativeAggregate Contract

Method	Description
initialize	Used when: AggregateProcessor is initialized (for window aggregate functions) AggregationIterator, ObjectAggregationIterator, TungstenAggregationIterator (for aggregate functions)
inputAggBufferOffset
merge	Used when: AggregationIterator does generateProcessRow (for aggregate functions)
mutableAggBufferOffset
update	Used when: AggregateProcessor is updated (for window aggregate functions) AggregationIterator (for aggregate functions)
withNewInputAggBufferOffset
withNewMutableAggBufferOffset