File based driver properties

Note: This feature is available in Enterprise, AWS, Team editions only.

Table of contents

• Overview
• General properties
• CSV driver properties
• XLSX driver properties
• JSON driver properties
• XML driver properties
• Parquet driver properties

Overview

File drivers allow you to interact with various file formats, like CSV, Excel, JSON, XML, and Parquet, as if they were databases. Each driver comes with customizable properties to control how data is read and processed.

General properties

Property	Description	Default value
defaultSchema	Default schema name.	default
scanSubfolders	Scan subfolders for data files.	true
subfolderNameSeparator	Defines the separator used in subfolder names within schema names.	__
internalDbBatchSize	Internal database batch size.	1000
internalDbTransactionSize	A number of batches in a single transaction.	10
useInternalDb	Use internal database for complex queries.	true

CSV driver properties

Property	Description	Default value
separator	The delimiter to use for separating entries.	,
escapeChar	The character to use for escaping a separator or quote.	\
quoteChar	The character to use for quoted elements.	"
strictQuotes	Sets if characters outside the quotes are ignored.	false
ignoreLeadingWhitespace	If true, parser should ignore white space before a quote in a field.	true
ignoreQuotations	If true, treat quotations like any other character.	false
nullFieldIndicator	Which field content will be returned as null.	NEITHER
trimWhitespaces	If true, parser should trim whitespaces from the beginning and end of the field.	true
header	If true, the first row is treated as a header.	true
sampleRows	Number of rows to extract metadata from.	5
wildcard	Wildcard for file names.	**.{csv,tcv,txt}

XLSX driver properties

Property	Description	Default value
header	Indicates if the file has a header.	true
sampleRows	Number of rows to extract metadata from.	5
wildcard	Wildcard for file names.	**.{xlsx}
schemaNameMode	Defines how the schema name is formed:	RELATIVE_DIR_PATH
	- RELATIVE_DIR_PATH: Uses the concatenated relative path of the directory, and the table name is $file_name + _ + $sheet_name.
	- RELATIVE_FILE_PATH: Uses the relative file path to generate the schema name. Each sheet in the file becomes a table.

JSON driver properties

Property	Description	Default value
sampleRows	Number of rows to extract metadata from.	5
wildcard	Wildcard for file names.	**.{json}

XML driver properties

Property	Description	Default value
sampleRows	Number of rows to extract metadata from.	5
wildcard	Wildcard for file names.	**.{xml}

Parquet driver properties

Property	Description	Default value
tmpFolder	Temporary folder for storing downloaded Parquet files from Cloud Storages.
wildcard	Wildcard for file names.	**.{parquet}