Skip to content

Latest commit

 

History

History
46 lines (34 loc) · 1.27 KB

spark-sql-SupportsScanColumnarBatch.adoc

File metadata and controls

46 lines (34 loc) · 1.27 KB

SupportsScanColumnarBatch

SupportsScanColumnarBatch is the contract…​FIXME

package org.apache.spark.sql.sources.v2.reader;

public interface SupportsScanColumnarBatch extends DataSourceReader {
  // only required methods that have no implementation
  // the others follow
  List<DataReaderFactory<ColumnarBatch>> createBatchDataReaderFactories();
}
Note

SupportsScanColumnarBatch is an Evolving contract that is evolving towards becoming a stable API, but is not a stable API yet and can change from one feature release to another release.

In other words, using the contract is as treading on thin ice.
Table 1. (Subset of) SupportsScanColumnarBatch Contract
Method Description

createBatchDataReaderFactories

Used when…​FIXME
Note
 No custom SupportsScanColumnarBatch are part of Spark 2.3.

enableBatchRead Method

default boolean enableBatchRead()

enableBatchRead flag is always enabled (i.e. true) unless overrode by custom SupportsScanColumnarBatches.

Note
 enableBatchRead is used when…​FIXME