spark-sql-RelationProvider.adoc

RelationProvider Contract — Relation Providers With Schema Inference

RelationProvider is the contract of BaseRelation providers that create a relation with schema inference.

Note	Schema inference is also called schema discovery.

The requirement of not specifying a user-defined schema or having one that does not match the relation is enforced when DataSource is requested for a BaseRelation for a given data source format. If specified and does not match, DataSource throws a AnalysisException:

[className] does not allow user-specified schemas.

package org.apache.spark.sql.sources

trait RelationProvider {
  def createRelation(
    sqlContext: SQLContext,
    parameters: Map[String, String]): BaseRelation
}

Table 1. RelationProvider Contract

Method	Description
createRelation	Creates a BaseRelation (for reading or writing) Used exclusively when DataSource is requested for a BaseRelation for a given data source format (and no user-defined schema or the user-defined schema matches schema of the BaseRelation)

Table 2. RelationProviders

RelationProvider	Description
JdbcRelationProvider
KafkaSourceProvider

Tip	Use SchemaRelationProvider for relation providers that require a user-defined schema.