From 886f071bbe9debf26a7a41f2f4f3d8738be5aaba Mon Sep 17 00:00:00 2001 From: Vik Nikolova Date: Thu, 29 Aug 2024 15:48:11 +0200 Subject: [PATCH] docs: Split up the "Deep dive into DSL" topic into several new topics (#2217) * split up the Deep dive into DSL topic intro subtopics, improve structure and formatting * Add an "About" topic and improve the table types and definition topics * Update the "Sequence" topic * Improve structure in the DSL related sub-topics and add descriptions to boolean operators * update table definition topics to highlight the core class and mention EntityID wrapper --- documentation-website/Writerside/c.list | 4 +- documentation-website/Writerside/hi.tree | 13 +- .../Writerside/topics/About.topic | 64 ++ .../Writerside/topics/Breaking-Changes.md | 2 +- .../topics/DSL-CRUD-operations.topic | 507 ++++++++++ .../Writerside/topics/DSL-Querying-data.topic | 414 ++++++++ .../Writerside/topics/DSL-Table-Types.topic | 51 + .../Writerside/topics/Data-Types.md | 717 -------------- .../Writerside/topics/Data-Types.topic | 859 +++++++++++++++++ .../Writerside/topics/Deep-Dive-into-DAO.md | 7 +- .../Writerside/topics/Deep-Dive-into-DSL.md | 912 ------------------ .../topics/Frequently-Asked-Questions.md | 6 +- .../topics/Getting-Started-with-Exposed.topic | 2 +- .../Writerside/topics/Home.topic | 4 +- .../Writerside/topics/SQL-Functions.md | 2 +- .../Writerside/topics/Table-Definition.md | 321 ------ .../Writerside/topics/Table-Definition.topic | 440 +++++++++ .../topics/Working-with-Schema.topic | 48 + .../topics/Working-with-Sequence.topic | 58 ++ 19 files changed, 2467 insertions(+), 1964 deletions(-) create mode 100644 documentation-website/Writerside/topics/About.topic create mode 100644 documentation-website/Writerside/topics/DSL-CRUD-operations.topic create mode 100644 documentation-website/Writerside/topics/DSL-Querying-data.topic create mode 100644 documentation-website/Writerside/topics/DSL-Table-Types.topic delete mode 100644 documentation-website/Writerside/topics/Data-Types.md create mode 100644 documentation-website/Writerside/topics/Data-Types.topic delete mode 100644 documentation-website/Writerside/topics/Deep-Dive-into-DSL.md delete mode 100644 documentation-website/Writerside/topics/Table-Definition.md create mode 100644 documentation-website/Writerside/topics/Table-Definition.topic create mode 100644 documentation-website/Writerside/topics/Working-with-Schema.topic create mode 100644 documentation-website/Writerside/topics/Working-with-Sequence.topic diff --git a/documentation-website/Writerside/c.list b/documentation-website/Writerside/c.list index c4bd745d05..a07260758e 100644 --- a/documentation-website/Writerside/c.list +++ b/documentation-website/Writerside/c.list @@ -3,4 +3,6 @@ SYSTEM "https://helpserver.labs.jb.gg/help/schemas/mvp/categories.dtd"> - \ No newline at end of file + + + diff --git a/documentation-website/Writerside/hi.tree b/documentation-website/Writerside/hi.tree index fc3e5ab45f..3686b9ea63 100644 --- a/documentation-website/Writerside/hi.tree +++ b/documentation-website/Writerside/hi.tree @@ -7,6 +7,7 @@ start-page="Home.topic"> + @@ -15,11 +16,17 @@ - - + + + + + + + + + - diff --git a/documentation-website/Writerside/topics/About.topic b/documentation-website/Writerside/topics/About.topic new file mode 100644 index 0000000000..272fa31969 --- /dev/null +++ b/documentation-website/Writerside/topics/About.topic @@ -0,0 +1,64 @@ + + + +

+ Exposed is a lightweight SQL library on top of a JDBC driver for the Kotlin programming language. It offers + two approaches for database access: the Domain-Specific Language (DSL) API and the Data Access Object (DAO) API. +

+

+ The Domain-Specific Language (DSL) API of Exposed provides a Kotlin-based abstraction for interacting with databases + . It closely mirrors actual SQL statements, allowing you to work + with familiar SQL concepts while benefiting from the type safety that Kotlin offers. +

+

+ The Data Access Object (DAO) API of Exposed provides an object-oriented approach for interacting with a database, + similar to traditional Object-Relational Mapping (ORM) frameworks like Hibernate. + This API is less verbose and provides a more intuitive and Kotlin-centric way to interact with your database. +

+

+ Exposed's flexibility allows you to choose the approach that best suits your project's needs, whether you + prefer the direct control of SQL with the DSL API or the higher-level abstraction of the DAO API. +

+

+ The official mascot of Exposed is the cuttlefish, which is well-known for its outstanding mimicry ability that enables it + to blend seamlessly into any environment. Similar to its mascot, Exposed can be used to mimic a variety of + database engines, which helps you to build applications without dependencies on any specific database engine + and to switch between them with very little or no changes. +

+ + Exposed currently supports the following databases: + +
  • + H2 (versions 2.x) +
    • +
  • + MariaDB +
    • +
  • + MySQL +
    • +
  • + Oracle +
    • +
  • + PostgreSQL (as well as PostgreSQL using the + pgjdbc-ng JDBC driver) +
    • +
  • + Microsoft SQL Server +
    • +
  • + SQLite +
    • +     +     + + + + + + +     diff --git a/documentation-website/Writerside/topics/Breaking-Changes.md b/documentation-website/Writerside/topics/Breaking-Changes.md index 1b0efa1c11..1810dcbe6e 100644 --- a/documentation-website/Writerside/topics/Breaking-Changes.md +++ b/documentation-website/Writerside/topics/Breaking-Changes.md @@ -184,7 +184,7 @@ integer("number").check { (it greaterEq 0) and (it lessEq Int.MAX_VALUE) } This applies to the specific `DateColumnType` in all 3 date/time modules and means `LocalDate` comparisons can now be done directly without conversions. * __H2, PostgreSQL__ Using `replace()` now throws an exception as the REPLACE command is not supported by these databases. If `replace()` was being used to perform an insert or update operation, all usages should instead be switched to `upsert()`. - [See documentation for UPSERT details](Deep-Dive-into-DSL.md#insert-or-update) + [See documentation for UPSERT details](DSL-CRUD-operations.topic#insert-or-update) * Operator classes `exists` and `notExists` have been renamed to `Exists` and `NotExists`. The functions `exists()` and `notExists()` have been introduced to return an instance of their respectively-named classes and to avoid unresolved reference issues. Any usages of these classes should be renamed to their capitalized forms. diff --git a/documentation-website/Writerside/topics/DSL-CRUD-operations.topic b/documentation-website/Writerside/topics/DSL-CRUD-operations.topic new file mode 100644 index 0000000000..ed60b38d65 --- /dev/null +++ b/documentation-website/Writerside/topics/DSL-CRUD-operations.topic @@ -0,0 +1,507 @@ + + + + + + +

    CRUD stands for Create Read Update Delete, which are four basic operations for a database to support. This + section shows how to perform SQL CRUD operations + using Kotlin DSL.

    + +

    Exposed provides several functions to insert rows into a table:

    + + + <code>insert</code> +

    To create a new table row, use the insert query. If the same row already exists in the table, + it throws an exception.

    + + Cities.insert { + it[name] = "St. Petersburg" + it[country] = Country.RUSSIA + it[population] = 500 + } + +

    The example corresponds to the following SQL statement:

    + + INSERT INTO CITIES (COUNTRY, "NAME", POPULATION) VALUES ('RUSSIA', 'St. Petersburg', 300) + +     + + + <code>insertAndGetId</code> + +

    Supported table types: IdTable()

    +     +

    To add a new row and return its ID, use insertAndGetId. If the same row already exists in + the table, it throws an exception.

    + + + val id = Cities.insertAndGetId { + it[name] = "St. Petersburg" + it[country] = Country.RUSSIA + it[population] = 500 + } + + + INSERT INTO CITIES (COUNTRY, "NAME", POPULATION) VALUES ('RUSSIA', 'St. Petersburg', 300) + +     + + + <code>insertIgnore</code> + +

    Supported on: MySQL, MariaDB, PostgreSQL, and SQLite

    +     +

    To allow insert statements to be executed without throwing any errors, use insertIgnore. + This may be useful, for example, when insertion conflicts are possible:

    + + + StarWarsFilms.insert { + it[sequelId] = 8 // column pre-defined with a unique index + it[name] = "The Last Jedi" + it[director] = "Rian Johnson" + } + + StarWarsFilms.insertIgnore { + it[sequelId] = 8 + it[name] = "The Rise of Skywalker" + it[director] = "JJ Abrams" + } + +

    + If insert was used instead of insertIgnore, this would throw a constraint violation exception + Instead, this new row is ignored and discarded. +

    +     + + + <code>insertIgnoreAndGetId</code> + +

    Supported on: MySQL, PostgreSQL, and SQLite

    +

    Table types: IntIdTable()

    +     +

    insertIgnoreAndGetId adds a new row and returns its ID. If the same row already + exists in the table, it ignores it and doesn't throw an exception.

    + + + val id = Cities.insertIgnoreAndGetId { + it[name] = "St. Petersburg" + it[country] = Country.RUSSIA + it[population] = 500 + } + + + INSERT IGNORE INTO CITIES (COUNTRY, "NAME", POPULATION) VALUES ('RUSSIA', 'St. Petersburg', 300) +     + + + <code>batchInsert</code> +

    batchInsert allows mapping a list of entities into table rows in a single SQL statement. + It is more efficient than using the insert query for each row as it initiates only one statement. +

    +

    The following example uses a simple list:

    + + + val cityNames = listOf("Paris", "Moscow", "Helsinki") + val allCitiesID = cities.batchInsert(cityNames) { name -> + this[cities.name] = name + } + + +

    Here is an example that uses a list of data class instances:

    + + + data class SWFilmData(val sequelId: Int, val name: String, val director: String) + + transaction { + // ... + val films = listOf( + SWFilmData(5, "The Empire Strikes Back", "Irvin Kershner"), + SWFilmData(4, "A New Hope", "George Lucas"), + SWFilmData(7, "The Force Awakens", "JJ Abrams") + ) + + StarWarsFilms.batchInsert(films) { (id, name, director) -> + this[StarWarsFilms.sequelId] = id + this[StarWarsFilms.name] = name + this[StarWarsFilms.director] = director + } + + StarWarsFilms.selectAll().count() // 3 + } + + + The batchInsert function will still create multiple INSERT statements when + interacting with your database. +

    To convert the INSERT statements into a BULK INSERT, use the + rewriteBatchedInserts=true + (or rewriteBatchedStatements=true) + option of your relevant JDBC driver.

    +

    For more information, see the documentation for this option for MySQL + and + PostgresSQL.

    + + +

    If you don't need to get the newly generated values, such as the auto-incremented ID, set the + shouldReturnGeneratedValues parameter to false. This increases the + performance of batch inserts by batching them in chunks, instead of always waiting for the database to + synchronize the newly inserted object state.

    +

    If you want to check if rewriteBatchedInserts and batchInsert are working + correctly, you need to enable JDBC logging for your driver. This is necessary, as Exposed will always + show the non-rewritten multiple inserts. For more information, see + how to enable logging in + PostgresSQL.

    + + + + + <code>select</code> +

    The select() function allows you to select specific columns or/and expressions.

    + + + val filmAndDirector = StarWarsFilms.select(StarWarsFilms.name, StarWarsFilms.director).map { + it[StarWarsFilms.name] to it[StarWarsFilms.director] + } + + +

    If you want to select only distinct value then use withDistinct() function:

    + + + val directors = + StarWarsFilms.select(StarWarsFilms.director).where { StarWarsFilms.sequelId less 5 }.withDistinct() + .map { + it[StarWarsFilms.director] + } + +     + + <code>selectAll</code> +

    To retrieve records from a table, use the selectAll() method:

    + + val query: Query = StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq 8 } + +

    Query inherits Iterable so it is possible to traverse it using map() + or forEach(): +

    + + + StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq 8 }.forEach { + println(it[StarWarsFilms.name]) + } + +     + +     + + +

    To update a record, use the update() function. + By default, it returns the number of updated rows. +

    + + val updatedCount = StarWarsFilms.update({ StarWarsFilms.name like "Episode%" }) { + it[StarWarsFilms.director] = "George Lucas" + } + +

    To update a column value using an expression, such as an increment, use update function + or setter:

    + + + StarWarsFilms.update({ StarWarsFilms.sequelId eq 8 }) { + it.update(StarWarsFilms.sequelId, StarWarsFilms.sequelId + 1) + // or + it[StarWarsFilms.sequelId] = StarWarsFilms.sequelId + 1 + } + + +     + + + <code>deleteWhere</code> +

    To delete records and return the count of deleted rows, use the deleteWhere function.

    + + StarWarsFilms.deleteWhere { StarWarsFilms.sequelId eq 8 } + +     + + <code>deleteIgnoreWhere</code> + +

    Supported on: MySQL and MariaDB

    +     +

    To delete records while ignoring any possible errors that occur during the process, use the + deleteIgnoreWhere function. The function will return the count of deleted rows. +

    + + StarWarsFilms.deleteIgnoreWhere { StarWarsFilms.sequelId eq 8 } + +     + + <code>deleteAll</code> +

    To deletes all rows in a table and return the count of deleted rows, use the deleteAll + function.

    + + StarWarsFilms.deleteAll { StarWarsFilms.sequelId eq 8 } + +     + +     + +

    Insert or update (Upsert) is a database operation that either inserts a new row or updates an existing row if + a duplicate constraint already exists. + The supported functionality of upsert() is dependent on the specific + database being used. + For example, MySQL's INSERT ... ON DUPLICATE KEY UPDATE statement automatically assesses the + primary key and unique indices + for a duplicate value, so using the function in Exposed would look like this:

    + + + // inserts a new row + StarWarsFilms.upsert { + it[sequelId] = 9 // column pre-defined with a unique index + it[name] = "The Rise of Skywalker" + it[director] = "Rian Johnson" + } + // updates existing row with the correct [director] + StarWarsFilms.upsert { + it[sequelId] = 9 + it[name] = "The Rise of Skywalker" + it[director] = "JJ Abrams" + } + +

    If none of the optional arguments are provided to upsert(), and an onUpdate() block + is omitted, the statements in the body block will be used for both the insert and update parts + of the operation. + This means that, for example, if a table mapping has columns with default values and these columns are + omitted from the body block, the default values will be + used for insertion as well as for the update operation.

    + + If the update operation should differ from the insert operation, then onUpdate() should be + provided an argument to set the specific columns to update, as seen in the example below. +

    If the update operation involves functions that should use the values that would have been inserted, then + these columns + should be marked using insertValue(), as seen in the example below.

    +     +

    Using another example, PostgreSQL allows more control over which key constraint columns to check for + conflict, whether different + values should be used for an update, and whether the update statement should have a WHERE + clause:

    + + + StarWarsFilms.upsert( + StarWarsFilms.sequelId, + onUpdate = { it[StarWarsFilms.sequelId] = StarWarsFilms.sequelId + 1 }, + where = { StarWarsFilms.director like stringLiteral("JJ%") } + ) { + it[sequelId] = 9 + it[name] = "The Rise of Skywalker" + it[director] = "JJ Abrams" + } + + StarWarsFilms.upsert( + onUpdate = { + it[StarWarsFilms.director] = concat(insertValue(StarWarsFilms.director), stringLiteral(" || "), StarWarsFilms.director) + } + ) { + it[sequelId] = 9 + it[name] = "The Rise of Skywalker" + it[director] = "Rian Johnson" + } + +

    If the update operation should be identical to the insert operation except for a few columns, + then onUpdateExclude should be provided an argument with the specific columns to exclude. + This parameter could also be used for the reverse case when only a small subset of columns should be updated + but duplicating the insert values is tedious:

    + + + // on conflict, all columns EXCEPT [director] are updated with values from the lambda block + StarWarsFilms.upsert(onUpdateExclude = listOf(StarWarsFilms.director)) { + it[sequelId] = 9 + it[name] = "The Rise of Skywalker" + it[director] = "JJ Abrams" + } + + // on conflict, ONLY column [director] is updated with value from the lambda block + StarWarsFilms.upsert( + onUpdateExclude = StarWarsFilms.columns - setOf(StarWarsFilms.director) + ) { + it[sequelId] = 9 + it[name] = "The Rise of Skywalker" + it[director] = "JJ Abrams" + } + +

    If a specific database supports user-defined key columns and none are provided, the table's primary key is + used. If there + is no defined primary key, the first unique index is used. If there are no unique indices, each database + handles this case + differently, so it is strongly advised that keys are defined to avoid unexpected results.

    + + Databases that do not support a specific Insert or Update command implement the standard MERGE INTO + ... USING statement with aliases and a derived table column list. + These include Oracle, SQL Server, and H2 compatibility modes (except for MySQL mode). + Any columns defined as key constraints (to be used in the ON clause) must be included in the + statement block to avoid throwing an error. + +     + +

    If you want to use the INSERT INTO ... SELECT SQL clause try the function Table.insert(Query): +

    + + + val substring = users.name.substring(1, 2) + cities.insert(users.select(substring).orderBy(users.id).limit(2)) + +

    By default, it will try to insert into all non auto-increment Table columns in the order they + are defined in the Table instance. If you want to specify columns or change the + order, provide a list of columns as the second parameter:

    + + + val userCount = users.selectAll().count() + users.insert( + users.select( + stringParam("Foo"), + Random().castTo<String>(VarCharColumnType()).substring(1, 10) + ), columns = listOf(users.name, users.id) + ) + +     + + +

    Supported on: SQLite, MySQL, and MariaDB

    +     +

    + The replace() + method acts in a similar manner to an upsert(). The only difference is that + if an insertion would violate a unique constraint, the existing row is deleted before the new row is inserted. +

    + + + object StarWarsFilms : Table() { + val sequelId: Column<Int> = integer("sequel_id").uniqueIndex() + val releaseYear: Column<Int> = integer("release_year") + val name: Column<String> = varchar("name", 50) + val director: Column<String> = varchar("director", 50) + val rating: Column<Double> = double("rating").default(10.0) + + override val primaryKey = PrimaryKey(sequelId, releaseYear) + } + + transaction { + // ... + // inserts a new row with default rating + StarWarsFilms.replace { + it[sequelId] = 9 + it[releaseYear] = 2019 + it[name] = "The Rise of Skywalker" + it[director] = "JJ Abrams" + } + // deletes existing row and inserts new row with set [rating] + StarWarsFilms.replace { + it[sequelId] = 9 + it[releaseYear] = 2019 + it[name] = "The Rise of Skywalker" + it[director] = "JJ Abrams" + it[rating] = 5.2 + } + } + +

    Unlike upsert(), none of the supporting databases allows a WHERE clause. + Also, the constraints used to assess a violation are limited to the primary key and unique indexes, so there + is no parameter for a custom key set.

    +

    The values specified in the statement block will be used for the insert statement, and any omitted columns + are set to their default values, if applicable.

    +

    + In the example above, if the original row was inserted with a user-defined rating and replace() + was executed with a block that omitted the rating column, + the newly inserted row would store the default rating value. This is because the old row was completely + deleted first. +

    +

    The REPLACE INTO ... SELECT SQL clause can be used by instead providing a query to replace(): +

    + + + val allRowsWithLowRating: Query = StarWarsFilms.selectAll().where { + StarWarsFilms.rating less 5.0 + } + StarWarsFilms.replace(allRowsWithLowRating) + +

    By default, it will try to insert into all non auto-increment Table columns in the order they + are defined in the Table instance. + If the columns need to be specified or the order should be changed, provide a list of columns as the second + parameter:

    + + + val oneYearLater = StarWarsFilms.releaseYear.plus(1) + val allRowsWithNewYear: Query = StarWarsFilms.select( + oneYearLater, StarWarsFilms.sequelId, StarWarsFilms.director, StarWarsFilms.name + ) + StarWarsFilms.replace( + allRowsWithNewYear, + columns = listOf( + StarWarsFilms.releaseYear, + StarWarsFilms.sequelId, + StarWarsFilms.director, + StarWarsFilms.name + ) + ) + +     + +

    Some databases return a count of the number of rows inserted, updated, or deleted by the CRUD operation. + For insert, upsert, and replace, this value can be accessed + using the statement class property insertedCount:

    + + + val insertStatement = StarWarsFilms.insertIgnore { + it[name] = "The Last Jedi" + it[sequelId] = 8 + it[director] = "Rian Johnson" + } + val rowCount: Int = insertStatement.insertedCount + +     + + +

    Supported on: PostgreSQL and SQLite

    +     +

    Some databases allow the return of additional data every time a row is either inserted, updated, or deleted. + This can be accomplished by using one of the following functions: +

    + +
  • insertReturning()
    • +
  • upsertReturning()
    • +
  • updateReturning()
    • +
  • deleteReturning()
    • +     +

    + Each of them take a list of the required table columns + as an argument. If not provided, all table columns will be returned by default: +

    + + + object Projects : Table("projects") { + val title = varchar("title", 64) + val budget = integer("budget") + val created = datetime("created").defaultExpression(CurrentDateTime) + } + + // returns all table columns by default + val created: LocalDateTime = Projects.insertReturning { + it[title] = "Project A" + it[budget] = 100 + }.single()[Projects.created] + + val updatedBudgets: List<Int> = Projects.updateReturning(listOf(Projects.budget)) { + it[budget] = Projects.budget.times(5) + }.map { + it[Projects.budget] + } + + + Unlike the base variants of these CRUD operations, a ReturningStatement behaves like a + Query by also extending Iterable, + so it will not be run by the database until the first attempt to iterate over its results. + +     + diff --git a/documentation-website/Writerside/topics/DSL-Querying-data.topic b/documentation-website/Writerside/topics/DSL-Querying-data.topic new file mode 100644 index 0000000000..47eacd8647 --- /dev/null +++ b/documentation-website/Writerside/topics/DSL-Querying-data.topic @@ -0,0 +1,414 @@ + + + + + + Working with <code>where</code> expressions +

    The query expression where expects a boolean operator (Op<Boolean>). + We've split the allowed conditions in different categories:

    + +
  • Basic conditions.
    • +
  • Logical conditions.
    • +
  • Conditions that check for a match in a pattern.
    • +
  • Conditions that check for a match in a range.
    • +
  • Conditions that check for a match in a collection.
    • +     + + + + <code>eq</code> + Checks if an expression is equal to some value. + + + <code>neq</code> + Checks if an expression is not equal to some value. + + + <code>isNull()</code> + Returns true if this expression is null, false otherwise. + + + <code>isNotNull()</code> + Returns false if this expression is null, true otherwise. + + + <code>less</code> + Checks if an expression is less than some value. + + + <code>lessEq</code> + Checks if an expression is less than or equal to some value. + + + <code>greater</code> + Checks if an expression is greater than some value. + + + <code>greaterEq</code> + Checks if an expression is greater than or equal to some value. + + + <code>exists</code> + Checks if a subquery returns at least one row. + + + <code>notExists</code> + Checks if a subquery does not return any row. + + + <code>isDistinctFrom</code> + Checks if this expression is not equal to another value, with null treated as a comparable value. + + + <code>isNotDistinctFrom</code> + Checks if an expression is equal to another value, with null treated as a comparable value. + + + + + + + + <code>not</code> + Returns the inverse of an expression. + + + <code>and</code> + Returns the result of performing a logical and operation between two expressions. + + + <code>or</code> + Returns the result of performing a logical or operation between two expressions. + + + <code>andIfNotNull</code> + Returns the result of performing a logical and operation between two expressions if + the second one is not null. + + + <code>orIfNotNull</code> + Returns the result of performing a logical or operation between two expressions if + the second is not null. + + + <code>compoundAnd()</code> + Reduces a list to a single expression by performing an and operation between all the + expressions in the list. + + + <code>compoundOr()</code> + Reduces a list to a single expression by performing an or operation between all the + expressions in the list. + + + + + + + <code>like</code> + Checks if an expression matches the specified pattern. + + StarWarsFilms.selectAll().where { StarWarsFilms.name like "The %" } + + + + <code>notLike</code> + Checks if an expression doesn't match the specified pattern. + + + <code>regexp</code> + Checks if an expression matches a regular expression. + + StarWarsFilms.selectAll().where { StarWarsFilms.name regexp "^The(\\s\\w+){2}\$" } + + + + <code>match</code> + Checks whether an expression matches a given pattern based on a specific mode. +

    + Supported only on MySQL and MariaDB. +

    +     +     +     + + + + <code>between(from: T, to: T)</code> + Checks if an expression is between the values from and to. Returns true if the + expression is between the lower and upper range values (inclusive). Date and time values are also + supported as arguments. + + StarWarsFilms.selectAll().where { StarWarsFilms.sequelId.between(4, 6) } + + + + + + + + <code>inList</code> +

    + Checks if an expression is equal to any element from list. +

    + + StarWarsFilms.selectAll().where { StarWarsFilms.sequelId inList listOf(6, 4) } + +

    inList also accepts multiple expressions to check for equality, either as a + Pair or a Triple:

    + + + val topRated = listOf(5 to "Empire Strikes Back", 4 to "A New Hope") + StarWarsFilms.selectAll().where { + StarWarsFilms.sequelId to StarWarsFilms.name inList topRated + } + +     + + <code>notInList</code> + Checks if an expression is not equal to any element in the provided collection. + +     + +

    In addition to the IN operator, the ANY and ALL operators are + available with any preceding comparison operator:

    + + + StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq anyFrom(arrayOf(6, 4)) } + +

    anyFrom() and allFrom() also accept subqueries, tables, and array expressions + as arguments.

    +     +     + + Conditional <code>where</code> +

    It is a rather common case to have a query with a where clause that depends on some other code's + conditions. Moreover, independent or nested conditions could + make it more complicated to prepare such where clauses. +

    +

    + Let's imagine that we have a form on a website where a user can optionally filter "Star Wars" + films by a director and/or a sequel. +

    + + + val query = StarWarsFilms.selectAll() + directorName?.let { + query.andWhere { StarWarsFilms.director eq it } + } + sequelId?.let { + query.andWhere { StarWarsFilms.sequelId eq it } + } + +

    But what if we want to conditionally select from another table and join it only when a condition is true? + You have to use adjustColumnSet and adjustSelect functions, which allow to extend + and modify join and select parts of a query (see kdoc + on that functions):

    + + + actorName?.let { + query.adjustColumnSet { innerJoin(Actors, { StarWarsFilms.sequelId }, { Actors.sequelId }) } + .adjustSelect { select(fields + Actors.columns).set } + .andWhere { Actors.name eq actorName } + } + +     + + +

    count() is a method of Query that is used like in the example below:

    + + + val count = StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq 8 }.count() + +     + +

    orderBy() accepts a list of columns mapped to boolean indicates if sorting should be ascending or descending. + Example:

    + + + StarWarsFilms.selectAll().orderBy(StarWarsFilms.sequelId to SortOrder.ASC) + +     + +

    In groupBy, define fields and their functions by the select() + method.

    + + + StarWarsFilms + .select(StarWarsFilms.sequelId.count(), StarWarsFilms.director) + .groupBy(StarWarsFilms.director) + +

    Available field functions are: count, sum, average, min + and max.

    +     +     + +

    You can use the limit function to prevent loading large data sets or use it for pagination with second offset + parameter.

    + + + // Take 2 films after the first one. + StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq Actors.sequelId }.limit(2, offset = 1) + +     + + +

    For the join examples below, consider the following tables:

    + + + object StarWarsFilms : IntIdTable() { + val sequelId: Column<Int> = integer("sequel_id").uniqueIndex() + val name: Column<String> = varchar("name", 50) + val director: Column<String> = varchar("director", 50) + } + object Actors : IntIdTable() { + val sequelId: Column<Int> = integer("sequel_id").uniqueIndex() + val name: Column<String> = varchar("name", 50) + } + object Roles : Table() { + val sequelId: Column<Int> = integer("sequel_id") + val actorId: Column<EntityID<Int>> = reference("actor_id", Actors) + val characterName: Column<String> = varchar("name", 50) + } + +

    Join to count how many actors star in each movie:

    + + + Actors.join(StarWarsFilms, JoinType.INNER, onColumn = Actors.sequelId, otherColumn = StarWarsFilms.sequelId) + .select(Actors.name.count(), StarWarsFilms.name) + .groupBy(StarWarsFilms.name) + +

    Instead of specifying onColumn and otherColumn, additionalConstraint + can be used (and allows specifying + other types of join conditions).

    + + + Actors.join( + StarWarsFilms, + JoinType.INNER, + additionalConstraint = { StarWarsFilms.sequelId eq Actors.sequelId }) + .select(Actors.name.count(), StarWarsFilms.name) + .groupBy(StarWarsFilms.name) + +

    When joining on a foreign key, the more concise innerJoin can be used:

    + + + (Actors innerJoin Roles) + .select(Roles.characterName.count(), Actors.name) + .groupBy(Actors.name) + .toList() + +

    This is equivalent to the following:

    + + + Actors.join(Roles, JoinType.INNER, onColumn = Actors.id, otherColumn = Roles.actorId) + .select(Roles.characterName.count(), Actors.name) + .groupBy(Actors.name) + .toList() + +     + +

    You can combine the results of multiple queries using .union(...). + Per the SQL specification, the queries must have the same number of columns, and not be marked for update. + Subqueries may be combined when supported by the database.

    + + + val lucasDirectedQuery = + StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.director eq "George Lucas" } + val abramsDirectedQuery = + StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.director eq "J.J. Abrams" } + val filmNames = lucasDirectedQuery.union(abramsDirectedQuery).map { it[StarWarsFilms.name] } + +

    Only unique rows are returned by default. Duplicates may be returned using .unionAll().

    + + + val lucasDirectedQuery = + StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.director eq "George Lucas" } + val originalTrilogyQuery = + StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.sequelId inList (3..5) } + val filmNames = lucasDirectedQuery.unionAll(originalTrilogyQuery).map { it[StarWarsFilms.name] } + +     +     + +

    Aliases allow preventing ambiguity between field names and table names. + Use the aliased var instead of original one:

    + + + val filmTable1 = StarWarsFilms.alias("ft1") + filmTable1.selectAll() // can be used in joins etc' + +

    Also, aliases allow you to use the same table in a join multiple times:

    + + + val sequelTable = StarWarsFilms.alias("sql") + val originalAndSequelNames = StarWarsFilms + .innerJoin(sequelTable, { StarWarsFilms.sequelId }, { sequelTable[StarWarsFilms.id] }) + .select(StarWarsFilms.name, sequelTable[StarWarsFilms.name]) + .map { it[StarWarsFilms.name] to it[sequelTable[StarWarsFilms.name]] } + +

    And they can be used when selecting from sub-queries:

    + + + val starWarsFilms = StarWarsFilms + .select(StarWarsFilms.id, StarWarsFilms.name) + .alias("swf") + val id = starWarsFilms[StarWarsFilms.id] + val name = starWarsFilms[StarWarsFilms.name] + starWarsFilms + .select(id, name) + .map { it[id] to it[name] } + +     + +

    A Query instance, which can be instantiated by calling selectAll() or select() + on a Table or Join, has many extension functions for building complex queries. + Some of these have already been mentioned above, like where(), groupBy(), and orderBy(). +

    +

    If a SELECT query with a special clause is required, a custom extension function can be + implemented to enable its use with other standard queries.

    +

    For example, MySQL index hints, which follow the table name in SQL, can be implemented on a + SELECT query by using the following custom function and class:

    + + + fun Query.indexHint(hint: String) = IndexHintQuery(this, hint) + + class IndexHintQuery( + val source: Query, + val indexHint: String + ) : Query(source.set, source.where) { + + init { + // copies any stored properties from the original query + source.copyTo(this) + } + + override fun prepareSQL(builder: QueryBuilder): String { + val originalSql = super.prepareSQL(builder) + val fromTableSql = " FROM ${transaction.identity(set.source as Table)} " + return originalSql.replace(fromTableSql, "$fromTableSql$indexHint ") + } + + override fun copy(): IndexHintQuery = IndexHintQuery(source.copy(), indexHint).also { copy -> + copyTo(copy) + } + } + + transaction { + val originalQuery = StarWarsFilms + .selectAll() + .withDistinct() + .where { StarWarsFilms.sequelId less 5 } + .groupBy(StarWarsFilms.director) + + val queryWithHint = originalQuery + .indexHint("FORCE INDEX (PRIMARY)") + .orderBy(StarWarsFilms.sequelId) + } + +     +     diff --git a/documentation-website/Writerside/topics/DSL-Table-Types.topic b/documentation-website/Writerside/topics/DSL-Table-Types.topic new file mode 100644 index 0000000000..05a8348f33 --- /dev/null +++ b/documentation-website/Writerside/topics/DSL-Table-Types.topic @@ -0,0 +1,51 @@ + + + + + + + Convenience <code>IdTable</code> class +

    + While Table is the foundational class for defining tables, Exposed also provides the base + IdTable convenience class. This class extends Table and is designed to simplify + the definition of tables that use a standard id column as the primary key. +

    +

    + For example, to provide an auto-incrementing id column of type Int, you can use + the IntIdTable subtype: +

    + + object StarWarsFilms : IntIdTable() { + val sequelId = integer("sequel_id").uniqueIndex() + val name = varchar("name", 50) + val director = varchar("director", 50) + } + + + It is important to note that the id column in this case is of type + Column<EntityID<Int>>. This means that to access the id column value, you + need to use the .value property: + + val idValue = StarWarsFilms.id.value + + + + EntityID is a wrapper class commonly associated with Exposed's DAO API, but it is also used in + IdTable classes within the DSL. This wrapper ensures that the id values are + handled consistently across the DSL and DAO APIs. +

    + For more information on EntityID and how it + integrates with the DAO API, see the DAO documentation. +

    +     +     + + + + + + +     diff --git a/documentation-website/Writerside/topics/Data-Types.md b/documentation-website/Writerside/topics/Data-Types.md deleted file mode 100644 index 8d99472974..0000000000 --- a/documentation-website/Writerside/topics/Data-Types.md +++ /dev/null @@ -1,717 +0,0 @@ - - -# Data Types - -Exposed supports the following data types in the table definition: -* `integer` - translates to DB `INT` -* `short` - translates to DB `SMALLINT` -* `long` - `BIGINT` -* `float` - `FLOAT` -* `decimal` - `DECIMAL` with scale and precision -* `bool` - `BOOLEAN` -* `char` - `CHAR` -* `varchar` - `VARCHAR` with length -* `text` - `TEXT` -* `enumeration` - `INT` ordinal value -* `enumerationByName` - `VARCHAR` -* `customEnumeration` - see [additional section](#how-to-use-database-enum-types) -* `blob` - `BLOB` -* `binary` - `VARBINARY` with length -* `uuid` - `BINARY(16)` -* `reference` - a foreign key -* `array` - `ARRAY` - -The `exposed-java-time` extension (`org.jetbrains.exposed:exposed-java-time:$exposed_version`) provides additional types: - -* `date` - `DATETIME` -* `time` - `TIME` -* `datetime` - `DATETIME` -* `timestamp` - `TIMESTAMP` -* `duration` - `DURATION` - - -Some types are different for specific DB dialect. - - -The `exposed-json` extension (`org.jetbrains.exposed:exposed-json:$exposed_version`) provides additional types -(see [how to use](#how-to-use-json-and-jsonb-types)): - -* `json` - `JSON` -* `jsonb` - `JSONB` - - -Databases store JSON values either in text or binary format, so Exposed provides two types to account for any potential -differences, if they exist, for example: - -- **PostgreSQL**: `json()` maps to `JSON`, while `jsonb()` maps to `JSONB`. -- **SQLite**: No native JSON type, so `json()` maps to TEXT, while `jsonb()` throws. -- **MySQL**: JSON type only supports binary format, so `json()` and `jsonb()` both map to JSON. -- **Oracle**: Exposed does not currently support the JSON binary format of Oracle 21c; only text format `json()` can be used. - - -## How to use database ENUM types -Some of the databases (e.g. MySQL, PostgreSQL, H2) support explicit ENUM types. Because keeping such columns in sync with -Kotlin enumerations using only JDBC metadata could be a huge challenge, Exposed doesn't provide a possibility to manage -such columns in an automatic way, but that doesn't mean that you can't use such column types. - -You have two options to work with ENUM database types and you should use `customEnumeration()` (available since version 0.10.3) in both cases: -1. Use an existing ENUM column from your table. In this case, the `sql` parameter in `customEnumeration()` can be left as `null`. -2. Create a new ENUM column using Exposed by providing the raw definition SQL to the `sql` parameter in `customEnumeration()`. - -As a JDBC driver can provide/expect specific classes for ENUM types, you must also provide from/to transformation functions for -them when defining a `customEnumeration`. - -For a class like `enum class Foo { BAR, BAZ }`, you can use the provided code below for your specific database: - -### MySQL, H2 -```Kotlin -val existingEnumColumn = customEnumeration("enumColumn", { value -> Foo.valueOf(value as String) }, { it.name }) -val newEnumColumn = customEnumeration("enumColumn", "ENUM('BAR', 'BAZ')", { value -> Foo.valueOf(value as String) }, { it.name }) -``` - -### PostgreSQL - -PostgreSQL requires that ENUM is defined as a separate type, so you have to create it before creating your table. -Also, the PostgreSQL JDBC driver returns `PGobject` instances for such values, so a `PGobject` with its type manually set to the ENUM type needs to be used for the `toDb` parameter. -The full working sample is provided below: -```Kotlin -class PGEnum>(enumTypeName: String, enumValue: T?) : PGobject() { - init { - value = enumValue?.name - type = enumTypeName - } -} - -object EnumTable : Table() { - val enumColumn = customEnumeration("enumColumn", "FooEnum", { value -> Foo.valueOf(value as String) }, { PGEnum("FooEnum", it) }) -} - -transaction { - exec("CREATE TYPE FooEnum AS ENUM ('BAR', 'BAZ');") - SchemaUtils.create(EnumTable) -} -``` - -## How to use Json and JsonB types - -Add the following dependencies to your `build.gradle.kts`: -```kotlin -val exposedVersion: String by project - -dependencies { - implementation("org.jetbrains.exposed:exposed-core:$exposedVersion") - implementation("org.jetbrains.exposed:exposed-json:$exposedVersion") -} -``` - -Exposed works together with the JSON serialization/deserialization library of your choice by allowing column definitions -that accept generic serializer and deserializer arguments: -```kotlin -fun json(name: String, serialize: (T) -> String, deserialize: (String) -> T): Column - -fun jsonb(name: String, serialize: (T) -> String, deserialize: (String) -> T): Column -``` - -Here's an example that leverages [kotlinx.serialization](https://github.com/Kotlin/kotlinx.serialization) to support -`@Serializable` classes. It uses a simpler form of `json()` that relies on the library's `KSerializer` interface: -```kotlin -@Serializable -data class Project(val name: String, val language: String, val active: Boolean) - -val format = Json { prettyPrint = true } - -object Teams : Table("team") { - val groupId = varchar("group_id", 32) - val project = json("project", format) // equivalent to json("project", format, Project.serializer()) -} - -transaction { - val mainProject = Project("Main", "Java", true) - Teams.insert { - it[groupId] = "A" - it[project] = mainProject - } - Teams.update({ Teams.groupId eq "A" }) { - it[project] = mainProject.copy(language = "Kotlin") - } - - Teams.selectAll().map { "Team ${it[Teams.groupId]} -> ${it[Teams.project]}" }.forEach { println(it) } - // Team A -> Project(name=Main, language=Kotlin, active=true) -} -``` - -Here's how the same `Project` and `Teams` would be defined using [Jackson](https://github.com/FasterXML/jackson) -with the `jackson-module-kotlin` dependency and the full form of `json()`: -```kotlin -val mapper = jacksonObjectMapper() - -data class Project(val name: String, val language: String, val active: Boolean) - -object Teams : Table("team") { - val groupId = varchar("group_id", 32) - val project = json("project", { mapper.writeValueAsString(it) }, { mapper.readValue(it) }) -} -``` - -### Json Functions - -JSON path strings can be used to extract values (either as JSON or as a scalar value) at a specific field/key: -```kotlin -val projectName = Teams.project.extract("name") -val languageIsKotlin = Teams.project.extract("language").lowerCase() eq "kotlin" -Teams.select(projectName).where { languageIsKotlin }.map { it[projectName] } -``` - - -Databases that support a path context root $ will have this value appended to the generated SQL path expression -by default, so it is not necessary to include it in the provided argument String. In the above example, if MySQL is being -used, the provided path arguments should be .name and .language respectively. - - -The JSON functions `exists()` and `contains()` are currently supported as well: -```kotlin -val hasActiveStatus = Teams.project.exists(".active") -val activeProjects = Teams.selectAll().where { hasActiveStatus }.count() - -// Depending on the database, filter paths can be provided instead, as well as optional arguments -// PostgreSQL example -val mainId = "Main" -val hasMainProject = Teams.project.exists(".name ? (@ == \$main)", optional = "{\"main\":\"$mainId\"}") -val mainProjects = Teams.selectAll().where { hasMainProject }.map { it[Teams.groupId] } - -val usesKotlin = Teams.project.contains("{\"language\":\"Kotlin\"}") -val kotlinTeams = Teams.selectAll().where { usesKotlin }.count() - -// Depending on the database, an optional path can be provided too -// MySQL example -val usesKotlin = Teams.project.contains("\"Kotlin\"", ".language") -val kotlinTeams = Teams.selectAll().where { usesKotlin }.count() -``` - -### Json Arrays - -JSON columns also accept JSON arrays as input values. For example, using the serializable data class `Project` from the -example above, the following details some ways to create such a column: - -```kotlin -object TeamProjects : Table("team_projects") { - val memberIds = json("member_ids", Json.Default) - val projects = json>("projects", Json.Default) - // equivalent to: - // @OptIn(ExperimentalSerializationApi::class) json("projects", Json.Default, ArraySerializer(Project.serializer())) -} - -transaction { - TeamProjects.insert { - it[memberIds] = intArrayOf(1, 2, 3) - it[projects] = arrayOf( - Project("A", "Kotlin", true), - Project("B", "Java", true) - ) - } - // generates SQL - // INSERT INTO team_projects (member_ids, projects) VALUES ([1,2,3], [{"name":"A","language":"Kotlin","active":true},{"name":"B","language":"Java","active":true}]) -} -``` - -## How to use Array types - -PostgreSQL and H2 databases support the explicit ARRAY data type. - -Exposed currently only supports columns defined as one-dimensional arrays, with the stored contents being any out-of-the-box or custom data type. -If the contents are of a type with a supported `ColumnType` in the `exposed-core` module, the column can be simply defined with that type: -```kotlin -object Teams : Table("teams") { - val memberIds = array("member_ids") - val memberNames = array("member_names") - val budgets = array("budgets") -} -``` - -If more control is needed over the base content type, or if the latter is user-defined or from a non-core module, the explicit type should be provided to the function: -```kotlin -object Teams : Table("teams") { - val memberIds = array("member_ids") - val memberNames = array("member_names", VarCharColumnType(colLength = 32)) - val deadlines = array("deadlines", KotlinLocalDateColumnType()).nullable() - val budgets = array("budgets") - val expenses = array("expenses", DoubleColumnType()).default(emptyList()) -} -``` -This will prevent an exception being thrown if Exposed cannot find an associated column mapping for the defined type. -Null array contents are allowed, and the explicit column type should be provided for these columns as well. - -An array column accepts inserts and retrieves stored array contents as a Kotlin `List`: -```kotlin -Teams.insert { - it[memberIds] = List(5) { UUID.randomUUID() } - it[memberNames] = List(5) { i -> "Member ${'A' + i}" } - it[budgets] = listOf(9999.0) -} -``` - -### Array Functions - -A single element in a stored array can be accessed using the index reference `get()` operator: -```kotlin -val firstMember = Teams.memberIds[1] -Teams - .select(firstMember) - .where { Teams.expenses[1] greater Teams.budgets[1] } -``` - -Both PostgreSQL and H2 use a one-based indexing convention, so the first element is retrieved by using index 1. - - -A new subarray can also be accessed by using `slice()`, which takes a lower and upper bound (inclusive): -```kotlin -Teams.select(Teams.deadlines.slice(1, 3)) -``` -Both arguments for these bounds are optional if using PostgreSQL. - -An array column can also be used as an argument for the `ANY` and `ALL` SQL operators, either by providing the entire column or a new array expression via `slice()`: -```kotlin -Teams - .selectAll() - .where { Teams.budgets[1] lessEq allFrom(Teams.expenses) } - -Teams - .selectAll() - .where { stringParam("Member A") eq anyFrom(Teams.memberNames.slice(1, 4)) } -``` - -## Custom Data Types - -If a database-specific data type is not immediately supported by Exposed, any existing and open column type class can be extended or -a custom `ColumnType` class can be implemented to achieve the same functionality. - -The following examples describe different ways to customize a column type, register a column with the custom type, -and then start using it in transactions. - -### Hierarchical tree-like data - -PostgreSQL provides a data type, [`ltree`](https://www.postgresql.org/docs/current/ltree.html), to represent hierarchical tree-like data. - -The hierarchy labels are stored as strings, so the existing `StringColumnType` class be extended with a few overrides: -```kotlin -import org.postgresql.util.PGobject - -class LTreeColumnType : StringColumnType() { - override fun sqlType(): String = "LTREE" - - override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { - val parameterValue: PGobject? = value?.let { - PGobject().apply { - type = sqlType() - this.value = value as? String - } - } - super.setParameter(stmt, index, parameterValue) - } -} -``` - -> When setting an object in a prepared statement with JDBC, any unknown data type without a JDBC mapping is set as a varying character string. -> To avoid a casting exception due to PostgreSQL's stricter type system, the type of the set parameter should be manually declared, -> by using a `PGobject` in `setParamater()`, as shown in the example above. -{style="note"} - -A table extension function can then be added to register a new column with this type: -```kotlin -fun Table.ltree(name: String): Column = registerColumn(name, LTreeColumnType()) - -object TestTable : Table("test_table") { - val path = ltree("path") - - init { - index(customIndexName = "path_gist_idx", indexType = "GIST", columns = arrayOf(path)) - index(customIndexName = "path_idx", indexType = "BTREE", columns = arrayOf(path)) - } -} -``` - -> To use the `ltree` data type, the extension must first be enabled in the database by running `exec("CREATE EXTENSION ltree;")`. -{style="note"} - -String values representing hierarchy labels can then be inserted and queried from the `path` column. -The following block shows an update of all records that have a stored `path` either equal to or a descendant of the path `Top.Science`, -by setting a subpath of the first 2 labels as the updated value: -```kotlin -transaction { - TestTable.update( - where = { TestTable.path isDescendantOrEq "Top.Science" } - ) { - it[path] = path.subltree(0, 2) - } -} - -fun Expression.subltree(start: Int, end: Int) = - CustomStringFunction("SUBLTREE", this, intParam(start), intParam(end)) - -infix fun ExpressionWithColumnType.isDescendantOrEq(other: T) = - IsDescendantOrEqOp(this, wrap(other)) - -class IsDescendantOrEqOp( - left: Expression, - right: Expression -) : ComparisonOp(left, right, "<@") -``` - -### Date and time data - -MySQL and MariaDB provide a data type, [`YEAR`](https://dev.mysql.com/doc/refman/8.4/en/year.html), for 1-byte storage of year values in the range of 1901 to 2155. - -This example assumes that the column accepts string input values, but a numerical format is also possible, in which case -`IntegerColumnType` could be extended instead: -```kotlin -class YearColumnType : StringColumnType(), IDateColumnType { - override fun sqlType(): String = "YEAR" - - override val hasTimePart: Boolean = false - - override fun valueFromDB(value: Any): String = when (value) { - is java.sql.Date -> value.toString().substringBefore('-') - else -> error("Retrieved unexpected value of type ${value::class.simpleName}") - } -} - -fun Table.year(name: String): Column = registerColumn(name, YearColumnType()) -``` - -The `IDateColumnType` interface is implemented to ensure that any default expressions are handled appropriately. For example, -a new object `CurrentYear` can be added as a default to avoid issues with the strict column typing: -```kotlin -object CurrentYear : Function(YearColumnType()) { - override fun toQueryBuilder(queryBuilder: QueryBuilder) { - queryBuilder { +"CURRENT_DATE" } - } -} - -object TestTable : Table("test_table") { - val established = year("established").defaultExpression(CurrentYear) -} -``` - -String values of different formats (depending on the enabled `sql_mode`) can then be inserted and queried from the `year` column: -```kotlin -transaction { - // disable strict mode to allow truncation of full date strings - exec("SET sql_mode=''") - - val yearData = listOf("1901", "2000", "2023-08-22", "2155") - TestTable.batchInsert(yearData) { year -> - this[TestTable.established] = year - } - - TestTable - .selectAll() - .where { TestTable.established less CurrentYear } - .toList() -} -``` - -### Ranges of data - -PostgreSQL provides multiple [range data types](https://www.postgresql.org/docs/16/rangetypes.html) of different subtypes. - -If more than one range subtype needs to be used, a base `RangeColumnType` class could be first introduced with the minimum common logic: -```kotlin -import org.postgresql.util.PGobject - -abstract class RangeColumnType, R : ClosedRange>( - val subType: ColumnType, -) : ColumnType() { - abstract fun List.toRange(): R - - override fun nonNullValueToString(value: R): String { - return "[${value.start},${value.endInclusive}]" - } - - override fun nonNullValueAsDefaultString(value: R): String { - return "'${nonNullValueToString(value)}'" - } - - override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { - val parameterValue: PGobject? = value?.let { - PGobject().apply { - type = sqlType() - this.value = nonNullValueToString(it as R) - } - } - super.setParameter(stmt, index, parameterValue) - } - - override fun valueFromDB(value: Any): R? = when (value) { - is PGobject -> value.value?.let { - val components = it.trim('[', ')').split(',') - components.toRange() - } - else -> error("Retrieved unexpected value of type ${value::class.simpleName}") - } -} -``` - -A class for the type `int4range` that accepts `IntRange` values could then be implemented: -```kotlin -class IntRangeColumnType : RangeColumnType(IntegerColumnType()) { - override fun sqlType(): String = "INT4RANGE" - - override fun List.toRange(): IntRange { - return IntRange(first().toInt(), last().toInt() - 1) - } -} - -fun Table.intRange(name: String): Column = registerColumn(name, IntRangeColumnType()) -``` - -If a custom Kotlin implementation for a `DateRange` is set up (using `Iterable` and `ClosedRange`), -then a class for the type `daterange` can also be added. This implementation would require a dependency on `exposed-kotlin-datetime`: -```kotlin -class DateRangeColumnType : RangeColumnType(KotlinLocalDateColumnType()) { - override fun sqlType(): String = "DATERANGE" - - override fun List.toRange(): DateRange { - val endInclusive = LocalDate.parse(last()).minus(1, DateTimeUnit.DAY) - return DateRange(LocalDate.parse(first()), endInclusive) - } -} - -fun Table.dateRange(name: String): Column = registerColumn(name, DateRangeColumnType()) -``` - -These new column types can be used in a table definition: -```kotlin -object TestTable : Table("test_table") { - val amounts = intRange("amounts").default(1..10) - val holidays = dateRange("holidays") -} -``` - -With the addition of some custom functions, the stored data can then be queried to return the upper bound of the date range -for all records that have an integer range within the specified bounds: -```kotlin -transaction { - val holidayEnd = TestTable.holidays.upperBound() - TestTable - .select(holidayEnd) - .where { TestTable.amounts isContainedBy 0..100 } - .toList() -} - -fun , CR : ClosedRange, R : CR?> ExpressionWithColumnType.upperBound() - = CustomFunction("UPPER", (columnType as RangeColumnType).subType, this) - -infix fun ?> ExpressionWithColumnType.isContainedBy(other: R) = - RangeIsContainedOp(this, wrap(other)) - -class RangeIsContainedOp?>( - left: Expression, - right: Expression -) : ComparisonOp(left, right, "<@") -``` - -### Predefined string data - -MySQL and MariaDB provide a data type, [`SET`](https://dev.mysql.com/doc/refman/8.4/en/set.html), -for strings that can have zero or more values from a defined list of permitted values. -This could be useful, for example, when storing a list of Kotlin enum constants. - -To use this type, a new `ColumnType` could be implemented with all the necessary overrides. This example instead takes advantage of -the existing logic in `StringColumnType` as the base for database storage, then uses a custom `ColumnTransformer` to achieve the final -transformation between a set of enum constants and a string: -```kotlin -class SetColumnType>( - private val enumClass: KClass -) : StringColumnType() { - // uses reflection to retrieve elements of the enum class - private val enumConstants by lazy { - enumClass.java.enumConstants?.map { it.name } ?: emptyList() - } - - override fun sqlType(): String = enumConstants - .takeUnless { it.isEmpty() } - ?.let { "SET(${it.joinToString { e -> "'$e'" }})" } - ?: error("SET column must be defined with a list of permitted values") -} - -inline fun > Table.set(name: String): Column = - registerColumn(name, SetColumnType(T::class)) - -class EnumListColumnType>( - private val enumClass: KClass -) : ColumnTransformer> { - private val enumConstants by lazy { - enumClass.java.enumConstants?.associateBy { it.name } ?: emptyMap() - } - - override fun unwrap(value: List): String { - return value.joinToString(separator = ",") { it.name } - } - - override fun wrap(value: String): List = value - .takeUnless { it.isEmpty() }?.let { - it.split(',').map { e -> - enumConstants[e] - ?: error("$it can't be associated with any value from ${enumClass.qualifiedName}") - } - } - ?: emptyList() -} -``` - -> See [column transformations](Deep-Dive-into-DSL.md#column-transformation) for more details about `ColumnTransformer`. -{style="note"} - -The new column type and transformer can then be used in a table definition: -```kotlin -enum class Vowel { A, E, I, O, U } - -object TestTable : Table("test_table") { - val vowel: Column> = set("vowel") - .transform(EnumListColumnType(Vowel::class)) - .default(listOf(Vowel.A, Vowel.E)) -} -``` - -Lists of enum constants can then be inserted and queried from the `set` column. The following block shows a query for all records that -have `Vowel.O` stored at any position in the `set` column string: -```kotlin -transaction { - TestTable.insert { it[vowel] = listOf(Vowel.U, Vowel.E) } - TestTable.insert { it[vowel] = emptyList() } - TestTable.insert { it[vowel] = Vowel.entries } - - TestTable - .selectAll() - .where { TestTable.vowel.findInSet(Vowel.O) greater 0 } - .toList() -} - -fun > Expression>.findInSet(enum: T) = - CustomFunction("FIND_IN_SET", IntegerColumnType(), stringParam(enum.name), this) -``` - -### Key-Value pair data - -PostgreSQL provides a data type, [`hstore`](https://www.postgresql.org/docs/16/hstore.html), to store key-value data pairs in a single text string. - -The existing `StringColumnType` class can be extended with a few overrides: -```kotlin -import org.postgresql.util.PGobject - -class HStoreColumnType : TextColumnType() { - override fun sqlType(): String = "HSTORE" - - override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { - val parameterValue: PGobject? = value?.let { - PGobject().apply { - type = sqlType() - this.value = value as? String - } - } - super.setParameter(stmt, index, parameterValue) - } -} -``` - -A table extension function can then be added to register a new column with this type. -This example assumes that the input values will be of type `Map`, so `transform()` is used on the string column to handle parsing: -```kotlin -fun Table.hstore(name: String): Column = registerColumn(name, HStoreColumnType()) - -object TestTable : Table("test_table") { - val bookDetails = hstore("book_details").transform( - wrap = { - it.trim('{', '}').split(", ") - .associate { pair -> - pair.substringBefore("=") to pair.substringAfter("=") - } - }, - unwrap = { - it.entries.joinToString(separator = ",") { (k, v) -> - "\"$k\"=>\"$v\"" - } - } - ) -} -``` - -> See [column transformations](Deep-Dive-into-DSL.md#column-transformation) for more details about `transform()`. -{style="note"} - -> To use the `hstore` data type, the extension must first be enabled in the database by running `exec("CREATE EXTENSION hstore;")`. -{style="note"} - -Map values representing key-value pairs of strings can then be inserted and queried from the `bookDetails` column. -The following block queries the value associated with the `title` key from all `bookDetails` records: -```kotlin -transaction { - TestTable.insert { - it[bookDetails] = mapOf( - "title" to "Kotlin in Action", - "edition" to "2" - ) - } - - val bookTitle = TestTable.bookDetails.getValue("title") - TestTable - .select(bookTitle) - .toList() -} - -fun > Expression.getValue(key: String) = - CustomOperator("->", TextColumnType(), this, stringParam(key)) -``` - -### Case insensitive data - -PostgreSQL provides a data type, [`citext`](https://www.postgresql.org/docs/16/citext.html), that represents a case-insensitive string type. - -The existing `StringColumnType` class can be extended with a few overrides: -```kotlin -import org.postgresql.util.PGobject - -class CitextColumnType( - colLength: Int -) : VarCharColumnType(colLength) { - override fun sqlType(): String = "CITEXT" - - override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { - val parameterValue: PGobject? = value?.let { - PGobject().apply { - type = sqlType() - this.value = value as? String - } - } - super.setParameter(stmt, index, parameterValue) - } -} -``` - -A table extension function can then be added to register a new column with this type: -```kotlin -fun Table.citext(name: String, length: Int): Column = - registerColumn(name, CitextColumnType(length)) - -object TestTable : Table("test_table") { - val firstName = citext("first_name", 32) -} -``` - -> To use the `citext` data type, the extension must first be enabled in the database by running `exec("CREATE EXTENSION citext;")`. -{style="note"} - -String values can then be inserted and queried from the `firstName` column in a case-insensitive manner: -```kotlin -transaction { - val allNames = listOf("Anna", "Anya", "Agna") - TestTable.batchInsert(allNames) { name -> - this[TestTable.firstName] = name - } - - TestTable - .selectAll() - .where { TestTable.firstName like "an%" } - .toList() -} -``` diff --git a/documentation-website/Writerside/topics/Data-Types.topic b/documentation-website/Writerside/topics/Data-Types.topic new file mode 100644 index 0000000000..e4832a6743 --- /dev/null +++ b/documentation-website/Writerside/topics/Data-Types.topic @@ -0,0 +1,859 @@ + + + + + +

    Exposed supports the following data types in the table definition:

    + +
  • integer - translates to DB INT
    • +
  • short - translates to DB SMALLINT
    • +
  • long - BIGINT
    • +
  • float - FLOAT
    • +
  • decimal - DECIMAL with scale and precision
    • +
  • bool - BOOLEAN
    • +
  • char - CHAR
    • +
  • varchar - VARCHAR with length
    • +
  • text - TEXT
    • +
  • enumeration - INT ordinal value
    • +
  • enumerationByName - VARCHAR
    • +
  • customEnumeration - see additional section
    • +
  • blob - BLOB
    • +
  • binary - VARBINARY with length
    • +
  • uuid - BINARY(16)
    • +
  • reference - a foreign key
    • +
  • array - ARRAY
    • + +

    The exposed-java-time extension + (org.jetbrains.exposed:exposed-java-time:$exposed_version) provides additional types:

    + +
  • date - DATETIME
    • +
  • time - TIME
    • +
  • datetime - DATETIME
    • +
  • timestamp - TIMESTAMP
    • +
  • duration - DURATION
    • +     + + Some types are different for specific DB dialect. + +

    The exposed-json extension (org.jetbrains.exposed:exposed-json:$exposed_version) + provides additional types + (see how to use):

    + +
  • json - JSON
    • +
  • jsonb - JSONB
    • +     + + Databases store JSON values either in text or binary format, so Exposed provides two types to account for any + potential + differences, if they exist, for example: + +
  • + PostgreSQL + : json() maps to JSON, while jsonb() maps to JSONB. +
    • +
  • + SQLite + : No native JSON type, so json() maps to TEXT, while jsonb() throws. +
    • +
  • + MySQL + : JSON type only supports binary format, so json() and jsonb() both map to + JSON. +
    • +
  • + Oracle + : Exposed does not currently support the JSON binary format of Oracle 21c; only text format + json() can be used. +
    • +     +     + +

    Some of the databases (e.g. MySQL, PostgreSQL, H2) support explicit ENUM types. Because keeping such columns + in sync with + Kotlin enumerations using only JDBC metadata could be a huge challenge, Exposed doesn't provide a + possibility to manage + such columns in an automatic way, but that doesn't mean that you can't use such column types.

    +

    You have two options to work with ENUM database types and you should use customEnumeration() + (available since version 0.10.3) in both cases:

    + +
  • Use an existing ENUM column from your table. In this case, the sql parameter in customEnumeration() + can be left as null. +
    • +
  • Create a new ENUM column using Exposed by providing the raw definition SQL to the sql + parameter in customEnumeration(). +
    • +     +

    As a JDBC driver can provide/expect specific classes for ENUM types, you must also provide from/to + transformation functions for + them when defining a customEnumeration.

    +

    For a class like enum class Foo { BAR, BAZ }, you can use the provided code below for your + specific database:

    + + + + val existingEnumColumn = customEnumeration("enumColumn", { value -> Foo.valueOf(value as String) }, { it.name }) + val newEnumColumn = customEnumeration("enumColumn", "ENUM('BAR', 'BAZ')", { value -> Foo.valueOf(value as String) }, { it.name }) + + + +

    PostgreSQL requires that ENUM is defined as a separate type, so you have to create it before creating + your table. + Also, the PostgreSQL JDBC driver returns PGobject instances for such values, so a PGobject + with its type manually set to the ENUM type needs to be used for the toDb parameter. + The full working sample is provided below:

    + + >(enumTypeName: String, enumValue: T?) : PGobject() { + init { + value = enumValue?.name + type = enumTypeName + } + } + + object EnumTable : Table() { + val enumColumn = customEnumeration("enumColumn", "FooEnum", { value -> Foo.valueOf(value as String) }, { PGEnum("FooEnum", it) }) + } + + transaction { + exec("CREATE TYPE FooEnum AS ENUM ('BAR', 'BAZ');") + SchemaUtils.create(EnumTable) + } + ]]> +     +     + +

    Add the following dependencies to your build.gradle.kts:

    + + + val exposedVersion: String by project + + dependencies { + implementation("org.jetbrains.exposed:exposed-core:$exposedVersion") + implementation("org.jetbrains.exposed:exposed-json:$exposedVersion") + } + +

    Exposed works together with the JSON serialization/deserialization library of your choice by allowing column + definitions + that accept generic serializer and deserializer arguments:

    + + json(name: String, serialize: (T) -> String, deserialize: (String) -> T): Column + + fun jsonb(name: String, serialize: (T) -> String, deserialize: (String) -> T): Column + ]]> +

    Here's an example that leverages kotlinx.serialization + to support + @Serializable classes. It uses a simpler form of json() that relies on the + library's KSerializer interface:

    + + ("project", format) // equivalent to json("project", format, Project.serializer()) + } + + transaction { + val mainProject = Project("Main", "Java", true) + Teams.insert { + it[groupId] = "A" + it[project] = mainProject + } + Teams.update({ Teams.groupId eq "A" }) { + it[project] = mainProject.copy(language = "Kotlin") + } + + Teams.selectAll().map { "Team ${it[Teams.groupId]} -> ${it[Teams.project]}" }.forEach { println(it) } + // Team A -> Project(name=Main, language=Kotlin, active=true) + } + ]]> +

    Here's how the same Project and Teams would be defined using Jackson + with the jackson-module-kotlin dependency and the full form of json():

    + + (it) }) + } + ]]> + +

    JSON path strings can be used to extract values (either as JSON or as a scalar value) at a specific + field/key:

    + + ("name") + val languageIsKotlin = Teams.project.extract("language").lowerCase() eq "kotlin" + Teams.select(projectName).where { languageIsKotlin }.map { it[projectName] } + ]]> + + Databases that support a path context root $ will have this value appended to the generated + SQL path expression + by default, so it is not necessary to include it in the provided argument String. In the above example, + if MySQL is being + used, the provided path arguments should be .name and .language respectively. + +

    The JSON functions exists() and contains() are currently supported as well:

    + + + val hasActiveStatus = Teams.project.exists(".active") + val activeProjects = Teams.selectAll().where { hasActiveStatus }.count() + + // Depending on the database, filter paths can be provided instead, as well as optional arguments + // PostgreSQL example + val mainId = "Main" + val hasMainProject = Teams.project.exists(".name ? (@ == \$main)", optional = "{\"main\":\"$mainId\"}") + val mainProjects = Teams.selectAll().where { hasMainProject }.map { it[Teams.groupId] } + + val usesKotlin = Teams.project.contains("{\"language\":\"Kotlin\"}") + val kotlinTeams = Teams.selectAll().where { usesKotlin }.count() + + // Depending on the database, an optional path can be provided too + // MySQL example + val usesKotlin = Teams.project.contains("\"Kotlin\"", ".language") + val kotlinTeams = Teams.selectAll().where { usesKotlin }.count() + +     + +

    JSON columns also accept JSON arrays as input values. For example, using the serializable data class + Project from the + example above, the following details some ways to create such a column:

    + + ("member_ids", Json.Default) + val projects = json>("projects", Json.Default) + // equivalent to: + // @OptIn(ExperimentalSerializationApi::class) json("projects", Json.Default, ArraySerializer(Project.serializer())) + } + + transaction { + TeamProjects.insert { + it[memberIds] = intArrayOf(1, 2, 3) + it[projects] = arrayOf( + Project("A", "Kotlin", true), + Project("B", "Java", true) + ) + } + // generates SQL + // INSERT INTO team_projects (member_ids, projects) VALUES ([1,2,3], [{"name":"A","language":"Kotlin","active":true},{"name":"B","language":"Java","active":true}]) + } + ]]> +     +     + +

    PostgreSQL and H2 databases support the explicit ARRAY data type.

    +

    Exposed currently only supports columns defined as one-dimensional arrays, with the stored contents being any + out-of-the-box or custom data type. + If the contents are of a type with a supported ColumnType in the exposed-core + module, the column can be simply defined with that type:

    + + + object Teams : Table("teams") { + val memberIds = array<UUID>("member_ids") + val memberNames = array<String>("member_names") + val budgets = array<Double>("budgets") + } + +

    If more control is needed over the base content type, or if the latter is user-defined or from a non-core + module, the explicit type should be provided to the function:

    + + + object Teams : Table("teams") { + val memberIds = array<UUID>("member_ids") + val memberNames = array<String>("member_names", VarCharColumnType(colLength = 32)) + val deadlines = array<LocalDate>("deadlines", KotlinLocalDateColumnType()).nullable() + val budgets = array<Double>("budgets") + val expenses = array<Double?>("expenses", DoubleColumnType()).default(emptyList()) + } + +

    This will prevent an exception being thrown if Exposed cannot find an associated column mapping for the + defined type. + Null array contents are allowed, and the explicit column type should be provided for these columns as + well.

    +

    An array column accepts inserts and retrieves stored array contents as a Kotlin List:

    + + + Teams.insert { + it[memberIds] = List(5) { UUID.randomUUID() } + it[memberNames] = List(5) { i -> "Member ${'A' + i}" } + it[budgets] = listOf(9999.0) + } + + +

    A single element in a stored array can be accessed using the index reference get() operator: +

    + + + val firstMember = Teams.memberIds[1] + Teams + .select(firstMember) + .where { Teams.expenses[1] greater Teams.budgets[1] } + + + Both PostgreSQL and H2 use a one-based indexing convention, so the first element is retrieved by using + index 1. + +

    A new subarray can also be accessed by using slice(), which takes a lower and upper bound + (inclusive):

    + + + Teams.select(Teams.deadlines.slice(1, 3)) + +

    Both arguments for these bounds are optional if using PostgreSQL.

    +

    An array column can also be used as an argument for the ANY and ALL SQL + operators, either by providing the entire column or a new array expression via slice():

    + + + Teams + .selectAll() + .where { Teams.budgets[1] lessEq allFrom(Teams.expenses) } + + Teams + .selectAll() + .where { stringParam("Member A") eq anyFrom(Teams.memberNames.slice(1, 4)) } + +     +     + +

    If a database-specific data type is not immediately supported by Exposed, any existing and open column type + class can be extended or + a custom ColumnType class can be implemented to achieve the same functionality.

    +

    The following examples describe different ways to customize a column type, register a column with the custom + type, + and then start using it in transactions.

    + +

    PostgreSQL provides a data type, ltree, to represent + hierarchical tree-like data.

    +

    The hierarchy labels are stored as strings, so the existing StringColumnType class be + extended with a few overrides:

    + + + import org.postgresql.util.PGobject + + class LTreeColumnType : StringColumnType() { + override fun sqlType(): String = "LTREE" + + override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { + val parameterValue: PGobject? = value?.let { + PGobject().apply { + type = sqlType() + this.value = value as? String + } + } + super.setParameter(stmt, index, parameterValue) + } + } + + + When setting an object in a prepared statement with JDBC, any unknown data type without + a JDBC mapping is set as a varying character string. + To avoid a casting exception due to PostgreSQL's stricter type system, the type of the set parameter + should be manually declared, + by using a PGobject in setParamater(), as shown in the example above. + +

    A table extension function can then be added to register a new column with this type:

    + = registerColumn(name, LTreeColumnType()) + + object TestTable : Table("test_table") { + val path = ltree("path") + + init { + index(customIndexName = "path_gist_idx", indexType = "GIST", columns = arrayOf(path)) + index(customIndexName = "path_idx", indexType = "BTREE", columns = arrayOf(path)) + } + } + ]]> + + To use the ltree data type, the extension must first be enabled in the + database by running exec("CREATE EXTENSION ltree;"). + +

    String values representing hierarchy labels can then be inserted and queried from the path + column. + The following block shows an update of all records that have a stored path either equal to + or a descendant of the path Top.Science, + by setting a subpath of the first 2 labels as the updated value:

    + + Expression.subltree(start: Int, end: Int) = + CustomStringFunction("SUBLTREE", this, intParam(start), intParam(end)) + + infix fun ExpressionWithColumnType.isDescendantOrEq(other: T) = + IsDescendantOrEqOp(this, wrap(other)) + + class IsDescendantOrEqOp( + left: Expression, + right: Expression + ) : ComparisonOp(left, right, "<@") + ]]> +     + +

    MySQL and MariaDB provide a data type, YEAR, + for 1-byte storage of year values in the range of 1901 to 2155.

    +

    This example assumes that the column accepts string input values, but a numerical format is also + possible, in which case + IntegerColumnType could be extended instead:

    + + value.toString().substringBefore('-') + else -> error("Retrieved unexpected value of type ${value::class.simpleName}") + } + } + + fun Table.year(name: String): Column = registerColumn(name, YearColumnType()) + ]]> +

    The IDateColumnType interface is implemented to ensure that any default expressions are + handled appropriately. For example, + a new object CurrentYear can be added as a default to avoid issues with the strict column + typing:

    + + (YearColumnType()) { + override fun toQueryBuilder(queryBuilder: QueryBuilder) { + queryBuilder { +"CURRENT_DATE" } + } + } + + object TestTable : Table("test_table") { + val established = year("established").defaultExpression(CurrentYear) + } + ]]> +

    String values of different formats (depending on the enabled sql_mode) can then be inserted + and queried from the year column:

    + + + transaction { + // disable strict mode to allow truncation of full date strings + exec("SET sql_mode=''") + + val yearData = listOf("1901", "2000", "2023-08-22", "2155") + TestTable.batchInsert(yearData) { year -> + this[TestTable.established] = year + } + + TestTable + .selectAll() + .where { TestTable.established less CurrentYear } + .toList() + } + +     + +

    PostgreSQL provides multiple range data + types of different subtypes.

    +

    If more than one range subtype needs to be used, a base RangeColumnType class could be first + introduced with the minimum common logic:

    + + , R : ClosedRange>( + val subType: ColumnType, + ) : ColumnType() { + abstract fun List.toRange(): R + + override fun nonNullValueToString(value: R): String { + return "[${value.start},${value.endInclusive}]" + } + + override fun nonNullValueAsDefaultString(value: R): String { + return "'${nonNullValueToString(value)}'" + } + + override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { + val parameterValue: PGobject? = value?.let { + PGobject().apply { + type = sqlType() + this.value = nonNullValueToString(it as R) + } + } + super.setParameter(stmt, index, parameterValue) + } + + override fun valueFromDB(value: Any): R? = when (value) { + is PGobject -> value.value?.let { + val components = it.trim('[', ')').split(',') + components.toRange() + } + else -> error("Retrieved unexpected value of type ${value::class.simpleName}") + } + } + ]]> +

    A class for the type int4range that accepts IntRange values could then be + implemented:

    + + (IntegerColumnType()) { + override fun sqlType(): String = "INT4RANGE" + + override fun List.toRange(): IntRange { + return IntRange(first().toInt(), last().toInt() - 1) + } + } + + fun Table.intRange(name: String): Column = registerColumn(name, IntRangeColumnType()) + ]]> +

    If a custom Kotlin implementation for a DateRange is set up (using Iterable and + ClosedRange), + then a class for the type daterange can also be added. This implementation would require a + dependency on exposed-kotlin-datetime:

    + + (KotlinLocalDateColumnType()) { + override fun sqlType(): String = "DATERANGE" + + override fun List.toRange(): DateRange { + val endInclusive = LocalDate.parse(last()).minus(1, DateTimeUnit.DAY) + return DateRange(LocalDate.parse(first()), endInclusive) + } + } + + fun Table.dateRange(name: String): Column & lt;DateRange> = registerColumn(name, DateRangeColumnType()) + ]]> +

    These new column types can be used in a table definition:

    + + + object TestTable : Table("test_table") { + val amounts = intRange("amounts").default(1..10) + val holidays = dateRange("holidays") + } + +

    With the addition of some custom functions, the stored data can then be queried to return the upper bound + of the date range + for all records that have an integer range within the specified bounds:

    + + , CR : ClosedRange, R : CR?> ExpressionWithColumnType.upperBound() = + CustomFunction("UPPER", (columnType as RangeColumnType).subType, this) + + infix fun ?> ExpressionWithColumnType.isContainedBy(other: R) = + RangeIsContainedOp(this, wrap(other)) + + class RangeIsContainedOp?>( + left: Expression, + right: Expression + ) : ComparisonOp(left, right, "<@") + ]]> +     + +

    MySQL and MariaDB provide a data type, SET, + for strings that can have zero or more values from a defined list of permitted values. + This could be useful, for example, when storing a list of Kotlin enum constants.

    +

    To use this type, a new ColumnType could be implemented with all the necessary overrides. + This example instead takes advantage of + the existing logic in StringColumnType as the base for database storage, then uses a custom + ColumnTransformer to achieve the final + transformation between a set of enum constants and a string:

    + + >( + private val enumClass: KClass + ) : StringColumnType() { + // uses reflection to retrieve elements of the enum class + private val enumConstants by lazy { + enumClass.java.enumConstants?.map { it.name } ?: emptyList() + } + + override fun sqlType(): String = enumConstants + .takeUnless { it.isEmpty() } + ?.let { "SET(${it.joinToString { e -> "'$e'" }})" } + ?: error("SET column must be defined with a list of permitted values") + } + + inline fun > Table.set(name: String): Column = + registerColumn(name, SetColumnType(T::class)) + + class EnumListColumnType>( + private val enumClass: KClass + ) : ColumnTransformer> { + private val enumConstants by lazy { + enumClass.java.enumConstants?.associateBy { it.name } ?: emptyMap() + } + + override fun unwrap(value: List): String { + return value.joinToString(separator = ",") { it.name } + } + + override fun wrap(value: String): List = value + .takeUnless { it.isEmpty() }?.let { + it.split(',').map { e -> + enumConstants[e] + ?: error("$it can't be associated with any value from ${enumClass.qualifiedName}") + } + } + ?: emptyList() + } + ]]> + + See column + transformations for more details about ColumnTransformer. + +

    The new column type and transformer can then be used in a table definition:

    + + > = set("vowel") + .transform(EnumListColumnType(Vowel::class)) + .default(listOf(Vowel.A, Vowel.E)) + } + ]]> +

    Lists of enum constants can then be inserted and queried from the set column. The following + block shows a query for all records that + have Vowel.O stored at any position in the set column string:

    + + > Expression>.findInSet(enum: T) = + CustomFunction("FIND_IN_SET", IntegerColumnType(), stringParam(enum.name), this) + ]]> +     + +

    PostgreSQL provides a data type, hstore, to store key-value + data pairs in a single text string.

    +

    The existing StringColumnType class can be extended with a few overrides:

    + + + import org.postgresql.util.PGobject + + class HStoreColumnType : TextColumnType() { + override fun sqlType(): String = "HSTORE" + + override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { + val parameterValue: PGobject? = value?.let { + PGobject().apply { + type = sqlType() + this.value = value as? String + } + } + super.setParameter(stmt, index, parameterValue) + } + } + +

    A table extension function can then be added to register a new column with this type. + This example assumes that the input values will be of type Map<String, String>, so + transform() is used on the string column to handle parsing:

    + + + fun Table.hstore(name: String): Column<String> = registerColumn(name, HStoreColumnType()) + + object TestTable : Table("test_table") { + val bookDetails = hstore("book_details").transform( + wrap = { + it.trim('{', '}').split(", ") + .associate { pair -> + pair.substringBefore("=") to pair.substringAfter("=") + } + }, + unwrap = { + it.entries.joinToString(separator = ",") { (k, v) -> + "\"$k\"=>\"$v\"" + } + } + ) + } + + + See column + transformations for more details about transform(). + + + To use the hstore data type, the extension must first be enabled in the + database by running exec("CREATE EXTENSION hstore;"). + +

    Map values representing key-value pairs of strings can then be inserted and queried from the bookDetails + column. + The following block queries the value associated with the title key from all bookDetails + records:

    + + > Expression.getValue(key: String) = + CustomOperator("->", TextColumnType(), this, stringParam(key)) + ]]> +     + +

    PostgreSQL provides a data type, citext, that represents a + case-insensitive string type.

    +

    The existing StringColumnType class can be extended with a few overrides:

    + + + import org.postgresql.util.PGobject + + class CitextColumnType( + colLength: Int + ) : VarCharColumnType(colLength) { + override fun sqlType(): String = "CITEXT" + + override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) { + val parameterValue: PGobject? = value?.let { + PGobject().apply { + type = sqlType() + this.value = value as? String + } + } + super.setParameter(stmt, index, parameterValue) + } + } + +

    A table extension function can then be added to register a new column with this type:

    + + + fun Table.citext(name: String, length: Int): Column<String> = + registerColumn(name, CitextColumnType(length)) + + object TestTable : Table("test_table") { + val firstName = citext("first_name", 32) + } + + +

    To use the citext data type, the extension must first be enabled in the + database by running exec("CREATE EXTENSION citext;").

    +     +

    String values can then be inserted and queried from the firstName column in a + case-insensitive manner:

    + + + transaction { + val allNames = listOf("Anna", "Anya", "Agna") + TestTable.batchInsert(allNames) { name -> + this[TestTable.firstName] = name + } + + TestTable + .selectAll() + .where { TestTable.firstName like "an%" } + .toList() + } + +     +     + +

    Column transformations allow to define custom transformations between database column types and application's + data types. + This can be particularly useful when you need to store data in one format but work with it in another format + within your application.

    +

    Consider the following example, where we define a table to store meal times and transform these times into + meal types:

    + + + enum class Meal { + BREAKFAST, + LUNCH, + DINNER + } + + object Meals : Table() { + val mealTime: Column<Meal> = time("meal_time") + .transform( + wrap = { + when { + it.hour < 10 -> Meal.BREAKFAST + it.hour < 15 -> Meal.LUNCH + else -> Meal.DINNER + } + }, + unwrap = { + when (it) { + Meal.BREAKFAST -> LocalTime(8, 0) + Meal.LUNCH -> LocalTime(12, 0) + Meal.DINNER -> LocalTime(18, 0) + } + } + ) + } + +

    The transform function is used to apply custom transformations to the mealTime + column:

    + +
  • The wrap function transforms the stored LocalTime values into + Meal enums. It checks the hour of the stored time and returns the corresponding meal type. +
    • +
  • The unwrap function transforms Meal enums back into LocalTime + values for storage in the database. +
    • +     +

    Transformation could be also defined as an implementation of ColumnTransformer interface and + reused among different tables:

    + + + class MealTimeTransformer : ColumnTransformer<LocalTime, Meal> { + override fun wrap(value: LocalTime): Meal = when { + value.hour < 10 -> Meal.BREAKFAST + value.hour < 15 -> Meal.LUNCH + else -> Meal.DINNER + } + + override fun unwrap(value: Meal): LocalTime = when (value) { + Meal.BREAKFAST -> LocalTime(8, 0) + Meal.LUNCH -> LocalTime(12, 0) + Meal.DINNER -> LocalTime(18, 0) + } + } + + object Meals : Table() { + val mealTime: Column<Meal> = time("meal_time").transform(MealTimeTransformer()) + } + +     + diff --git a/documentation-website/Writerside/topics/Deep-Dive-into-DAO.md b/documentation-website/Writerside/topics/Deep-Dive-into-DAO.md index e399ccdd60..688e89c038 100644 --- a/documentation-website/Writerside/topics/Deep-Dive-into-DAO.md +++ b/documentation-website/Writerside/topics/Deep-Dive-into-DAO.md @@ -118,7 +118,10 @@ val movies = StarWarsFilm.all() val movies = StarWarsFilm.find { StarWarsFilms.sequelId eq 8 } val movie = StarWarsFilm.findById(5) ``` -For a list of available predicates, see DSL Where expression. + +For a list of available predicates, see +DSL Where expression. + Read a value from a property similar to any property in a Kotlin class: ```kotlin @@ -408,7 +411,7 @@ class StarWarsFilm(id: EntityID) : IntEntity(id) { var director by Director referencedOn StarWarsFilms } ``` -For more information on creating table foreign key constraints, see DSL Foreign Key. +For more information on creating table foreign key constraints, see DSL Foreign Key. Now you can get the director for a `StarWarsFilm` object, `movie`, in the same way you would get any other field: ```kotlin diff --git a/documentation-website/Writerside/topics/Deep-Dive-into-DSL.md b/documentation-website/Writerside/topics/Deep-Dive-into-DSL.md deleted file mode 100644 index e76a1b075a..0000000000 --- a/documentation-website/Writerside/topics/Deep-Dive-into-DSL.md +++ /dev/null @@ -1,912 +0,0 @@ - - -# Deep Dive into DSL - -The DSL (Domain-Specific Language) API of Exposed is similar to actual SQL statements, but with the type safety that Kotlin offers. - -A database table is represented by an `object` inherited from `org.jetbrains.exposed.sql.Table` like this: - -```kotlin -object StarWarsFilms : Table() { - val id: Column = integer("id").autoIncrement() - val sequelId: Column = integer("sequel_id").uniqueIndex() - val name: Column = varchar("name", 50) - val director: Column = varchar("director", 50) - override val primaryKey = PrimaryKey(id, name = "PK_StarWarsFilms_Id") // PK_StarWarsFilms_Id is optional here -} -``` - -Tables that contains `Int` id with the name `id` can be declared like this: - -```kotlin -object StarWarsFilms : IntIdTable() { - val sequelId: Column = integer("sequel_id").uniqueIndex() - val name: Column = varchar("name", 50) - val director: Column = varchar("director", 50) -} -``` - -## CRUD operations - -CRUD stands for Create Read Update Delete, which are four basic operations for a database to support. This section shows how to perform SQL CRUD operations -using Kotlin DSL. - -### Create - -To create a new table row, you use the `insert` query. Exposed provides several functions to insert rows into a table: - -* `insert` adds a new row. If the same row already exists in the table, it throws an exception. - ```kotlin - // SQL: INSERT INTO CITIES (COUNTRY, "NAME", POPULATION) - // VALUES ('RUSSIA', 'St. Petersburg', 300) - Cities.insert { - it[name] = "St. Petersburg" - it[country] = Country.RUSSIA - it[population] = 500 - } - ``` -* `insertAndGetId` adds a new row and returns its ID. If the same row already exists in the table, it throws an exception. Works only with IntIdTable() tables. - ```kotlin - // SQL: INSERT INTO CITIES (COUNTRY, "NAME", POPULATION) - // VALUES ('RUSSIA', 'St. Petersburg', 300) - val id = Cities.insertAndGetId { - it[name] = "St. Petersburg" - it[country] = Country.RUSSIA - it[population] = 500 - } - ``` -* `insertIgnore` adds a new row. If the same row already exists in the table, it ignores it and doesn't throw an exception. This function is supported only for MySQL, - PostgreSQL, and SQLite. - ```kotlin - // SQL: INSERT IGNORE INTO CITIES (COUNTRY, "NAME", POPULATION) - // VALUES ('RUSSIA', 'St. Petersburg', 300) - Cities.insertIgnore { - it[name] = "St. Petersburg" - it[country] = Country.RUSSIA - it[population] = 500 - } - ``` -* `insertIgnoreAndGetId` adds a new row and returns its ID. If the same row already exists in the table, it ignores it and doesn't throw an exception. This function - is supported only for MySQL, PostgreSQL, and SQLite. Works only with `IntIdTable()` tables. - ```kotlin - // SQL: INSERT IGNORE INTO CITIES (COUNTRY, "NAME", POPULATION) - // VALUES ('RUSSIA', 'St. Petersburg', 300) - val id = Cities.insertIgnoreAndGetId { - it[name] = "St. Petersburg" - it[country] = Country.RUSSIA - it[population] = 500 - } - ``` - -Some databases return a count of the number of rows inserted, updated, or deleted by the CRUD operation. -For `insert()`, `upsert()`, and `replace()`, this value can be accessed using the statement class property, `insertedCount`: - -```kotlin -val insertStatement = StarWarsFilms.insertIgnore { - it[name] = "The Last Jedi" - it[sequelId] = 8 - it[director] = "Rian Johnson" -} -val rowCount: Int = insertStatement.insertedCount -``` - -### Read - -```kotlin -val query: Query = StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq 8 } -``` - -`Query` inherit `Iterable` so it is possible to traverse it with map/foreach etc'. For example: - -```kotlin -StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq 8 }.forEach { - println(it[StarWarsFilms.name]) -} -``` - -There is `select` function which allows you to select specific columns or/and expressions. - -```kotlin -val filmAndDirector = StarWarsFilms.select(StarWarsFilms.name, StarWarsFilms.director).map { - it[StarWarsFilms.name] to it[StarWarsFilms.director] -} -``` - -If you want to select only distinct value then use `withDistinct()` function: - -```kotlin -val directors = StarWarsFilms.select(StarWarsFilms.director).where { StarWarsFilms.sequelId less 5 }.withDistinct().map { - it[StarWarsFilms.director] -} -``` - -### Update - -```kotlin -StarWarsFilms.update({ StarWarsFilms.sequelId eq 8 }) { - it[StarWarsFilms.name] = "Episode VIII – The Last Jedi" -} -``` - -If you want to update column value with some expression like increment use `update` function or setter: - -```kotlin -StarWarsFilms.update({ StarWarsFilms.sequelId eq 8 }) { - with(SqlExpressionBuilder) { - it.update(StarWarsFilms.sequelId, StarWarsFilms.sequelId + 1) - // or - it[StarWarsFilms.sequelId] = StarWarsFilms.sequelId + 1 - } -} -``` - -If you want to know the number of updated rows, this value is also returned by the `update()` function: - -```kotlin -val updatedCount = StarWarsFilms.update({ StarWarsFilms.name like "Episode%" }) { - it[StarWarsFilms.director] = StarWarsFilms.director.upperCase() -} -``` - -### Delete - -```kotlin -StarWarsFilms.deleteWhere { StarWarsFilms.sequelId eq 8 } -``` - -Delete functions also return a count of the number of deleted rows, as for Update above. - -### Returning Data from Modified Rows - -Some databases (like PostgreSQL and SQLite) allow the return of additional data every time a row is either inserted, updated, or deleted by a CRUD operation. -This can be accomplished by using `insertReturning()`, `upsertReturning()`, `updateReturning()`, or `deleteReturning()` with a list of the required table columns -provided as an argument. If the latter is omitted, all table columns will be returned by default: - -```kotlin -object Projects : Table("projects") { - val title = varchar("title", 64) - val budget = integer("budget") - val created = datetime("created").defaultExpression(CurrentDateTime) -} - -// returns all table columns by default -val created: LocalDateTime = Projects.insertReturning { - it[title] = "Project A" - it[budget] = 100 -}.single()[Projects.created] - -val updatedBudgets: List = Projects.updateReturning(listOf(Projects.budget)) { - it[budget] = Projects.budget.times(5) -}.map { - it[Projects.budget] -} -``` - - -Unlike the base variants of these CRUD operations, a ReturningStatement behaves like a Query by also extending Iterable, -so it will not be run by the database until the first attempt to iterate over its results. - - -## Where expression - -Query expression (where) expects a boolean operator (ie: `Op`). -Allowed conditions are: - -``` -eq - (==) -neq - (!=) -isNull() -isNotNull() -less - (<) -lessEq - (<=) -greater - (>) -greaterEq - (>=) -like - (=~) -notLike - (!~) -exists -notExists -regexp -inList -notInList -between -match (MySQL MATCH AGAINST) -isDistinctFrom (null-safe equality comparison) -isNotDistinctFrom (null-safe equality comparison) -``` - -Allowed logical conditions are: - -``` -not -and -or -andIfNotNull -orIfNotNull -compoundAnd() -compoundOr() -``` - -## Conditional where - -It is a rather common case to have a query with a `where` clause that depends on some other code's conditions. Moreover, independent or nested conditions could -make it more complicated to prepare such `where` clauses. -Let's imagine that we have a form on a website where a user can optionally filter "Star Wars" films by a director and/or a sequel. -In Exposed version before 0.8.1 you had to code it like: - -```Kotlin -val condition = when { - directorName!=null && sequelId!=null -> - Op.build { StarWarsFilms.director eq directorName and (StarWarsFilms.sequelId eq sequelId) } - directorName!=null -> - Op.build { StarWarsFilms.director eq directorName } - sequelId!=null -> - Op.build { StarWarsFilms.sequelId eq sequelId } - else -> null -} -val query = condition?.let { StarWarsFilms.selectAll().where(condition) } ?: StarWarsFilms.selectAll() -``` - -or - -```Kotlin -val query = when { - directorName!=null && sequelId!=null -> - StarWarsFilms.selectAll().where { StarWarsFilms.director eq directorName and (StarWarsFilms.sequelId eq sequelId) } - directorName!=null -> - StarWarsFilms.selectAll().where { StarWarsFilms.director eq directorName } - sequelId!=null -> - StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq sequelId } - else -> StarWarsFilms.selectAll() -} -``` - -This is a very primitive example, but you should get the main idea about the problem. -Now let's try to write the same query in a more simple way (`andWhere` function available since 0.10.5): - -```Kotlin -val query = StarWarsFilms.selectAll() -directorName?.let { - query.andWhere { StarWarsFilms.director eq it } -} -sequelId?.let { - query.andWhere { StarWarsFilms.sequelId eq it } -} -``` - -But what if we want to conditionally select from another table and join it only when a condition is true? -You have to use `adjustColumnSet` and `adjustSelect` functions, which allow to extend and modify `join` and `select` parts of a query (see kdoc -on that functions): - -```Kotlin -actorName?.let { - query.adjustColumnSet { innerJoin(Actors, { StarWarsFilms.sequelId }, { Actors.sequelId }) } - .adjustSelect { select(fields + Actors.columns).set } - .andWhere { Actors.name eq actorName } -} -``` - -### Check for a match in a pattern - -```kotlin -StarWarsFilms.selectAll().where { StarWarsFilms.name like "The %" } -``` - -`notLike` is also available to check for expressions that do not match the provided pattern. - -To perform a pattern match that supports regular expressions, use `regexp` instead: - -```kotlin -StarWarsFilms.selectAll().where { StarWarsFilms.name regexp "^The(\\s\\w+){2}\$" } -``` - -### Check for a match in a range - -```kotlin -StarWarsFilms.selectAll().where { StarWarsFilms.sequelId.between(4, 6) } -``` - -The `between` operator returns `true` if the expression is between the lower and upper range values (inclusive). -Date and time values are also supported as arguments. - -### Check for a match in a collection - -```kotlin -StarWarsFilms.selectAll().where { StarWarsFilms.sequelId inList listOf(6, 4) } -``` - -`inList` also accepts multiple expressions to check for equality, either as a `Pair` or a `Triple`: - -```kotlin -val topRated = listOf(5 to "Empire Strikes Back", 4 to "A New Hope") -StarWarsFilms.selectAll().where { - StarWarsFilms.sequelId to StarWarsFilms.name inList topRated -} -``` - -`notInList` is available to check for expressions that are not equal to any elements in the provided collection. - -In addition to the `IN` operator, the `ANY` and `ALL` operators are available with any preceding comparison operator: -```kotlin -StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq anyFrom(arrayOf(6, 4)) } -``` - -`anyFrom()` and `allFrom()` also accept subqueries, tables, and array expressions as arguments. - -## Count - -`count()` is a method of `Query` that is used like in the example below: - -```kotlin -val count = StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq 8 }.count() -``` - -## Order-by - -Order-by accepts a list of columns mapped to boolean indicates if sorting should be ascending or descending. -Example: - -```kotlin -StarWarsFilms.selectAll().orderBy(StarWarsFilms.sequelId to SortOrder.ASC) -``` - -## Group-by - -In group-by, define fields and their functions (such as `count`) by the `select()` method. - -```kotlin -StarWarsFilms - .select(StarWarsFilms.sequelId.count(), StarWarsFilms.director) - .groupBy(StarWarsFilms.director) -``` - -Available functions are: - -``` -count -sum -max -min -average -... -``` - -## Limit - -You can use limit function to prevent loading large data sets or use it for pagination with second `offset` parameter. - -```kotlin -// Take 2 films after the first one. -StarWarsFilms.selectAll().where { StarWarsFilms.sequelId eq Actors.sequelId }.limit(2, offset = 1) -``` - -## Join - -For the join examples below, consider the following tables: - -```kotlin -object StarWarsFilms : IntIdTable() { - val sequelId: Column = integer("sequel_id").uniqueIndex() - val name: Column = varchar("name", 50) - val director: Column = varchar("director", 50) -} -object Actors : IntIdTable() { - val sequelId: Column = integer("sequel_id").uniqueIndex() - val name: Column = varchar("name", 50) -} -object Roles : Table() { - val sequelId: Column = integer("sequel_id") - val actorId: Column> = reference("actor_id", Actors) - val characterName: Column = varchar("name", 50) -} -``` - -Join to count how many actors star in each movie: - -```kotlin -Actors.join(StarWarsFilms, JoinType.INNER, onColumn = Actors.sequelId, otherColumn = StarWarsFilms.sequelId) - .select(Actors.name.count(), StarWarsFilms.name) - .groupBy(StarWarsFilms.name) -``` - -Instead of specifying `onColumn` and `otherColumn`, `additionalConstraint` can be used (and allows specifying -other types of join conditions). - -```kotlin -Actors.join(StarWarsFilms, JoinType.INNER, additionalConstraint = { StarWarsFilms.sequelId eq Actors.sequelId }) - .select(Actors.name.count(), StarWarsFilms.name) - .groupBy(StarWarsFilms.name) -``` - -When joining on a foreign key, the more concise `innerJoin` can be used: - -```kotlin -(Actors innerJoin Roles) - .select(Roles.characterName.count(), Actors.name) - .groupBy(Actors.name) - .toList() -``` - -This is equivalent to the following: - -```kotlin -Actors.join(Roles, JoinType.INNER, onColumn = Actors.id, otherColumn = Roles.actorId) - .select(Roles.characterName.count(), Actors.name) - .groupBy(Actors.name) - .toList() -``` - -## Union - -You can combine the results of multiple queries using `.union(...)`. -Per the SQL specification, the queries must have the same number of columns, and not be marked for update. -Subqueries may be combined when supported by the database. - -```kotlin -val lucasDirectedQuery = StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.director eq "George Lucas" } -val abramsDirectedQuery = StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.director eq "J.J. Abrams" } -val filmNames = lucasDirectedQuery.union(abramsDirectedQuery).map { it[StarWarsFilms.name] } -``` - -Only unique rows are returned by default. Duplicates may be returned using `.unionAll()`. - -```kotlin -val lucasDirectedQuery = StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.director eq "George Lucas" } -val originalTrilogyQuery = StarWarsFilms.select(StarWarsFilms.name).where { StarWarsFilms.sequelId inList (3..5) } -val filmNames = lucasDirectedQuery.unionAll(originalTrilogyQuery).map { it[StarWarsFilms.name] } -``` - -## Alias - -Aliases allow preventing ambiguity between field names and table names. -Use the aliased var instead of original one: - -```Kotlin -val filmTable1 = StarWarsFilms.alias("ft1") -filmTable1.selectAll() // can be used in joins etc' -``` - -Also, aliases allow you to use the same table in a join multiple times: - -```Kotlin -val sequelTable = StarWarsFilms.alias("sql") -val originalAndSequelNames = StarWarsFilms - .innerJoin(sequelTable, { StarWarsFilms.sequelId }, { sequelTable[StarWarsFilms.id] }) - .select(StarWarsFilms.name, sequelTable[StarWarsFilms.name]) - .map { it[StarWarsFilms.name] to it[sequelTable[StarWarsFilms.name]] } -``` - -And they can be used when selecting from sub-queries: - -```kotlin -val starWarsFilms = StarWarsFilms - .select(StarWarsFilms.id, StarWarsFilms.name) - .alias("swf") -val id = starWarsFilms[StarWarsFilms.id] -val name = starWarsFilms[StarWarsFilms.name] -starWarsFilms - .select(id, name) - .map { it[id] to it[name] } -``` - -## Schema - -You can create a schema or drop an existing one: - -```Kotlin -val schema = Schema("my_schema") // my_schema is the schema name. -// Creates a Schema -SchemaUtils.createSchema(schema) -// Drops a Schema -SchemaUtils.dropSchema(schema) -``` - -Also, you can specify the schema owner like this (some databases require the explicit owner) : - -```Kotlin -val schema = Schema("my_schema", authorization = "owner") -``` - -If you have many schemas and you want to set a default one, you can use: - -```Kotlin -SchemaUtils.setSchema(schema) -``` - -## Sequence - -If you want to use Sequence, Exposed allows you to: - -### Define a Sequence - -```Kotlin -val myseq = Sequence("my_sequence") // my_sequence is the sequence name. -``` - -Several parameters can be specified to control the properties of the sequence: - -```Kotlin -private val myseq = Sequence( - name = "my_sequence", - startWith = 4, - incrementBy = 2, - minValue = 1, - maxValue = 10, - cycle = true, - cache = 20 -) -``` - -### Create and Drop a Sequence - -```Kotlin -// Creates a sequence -SchemaUtils.createSequence(myseq) -// Drops a sequence -SchemaUtils.dropSequence(myseq) -``` - -### Use the NextVal function - -You can use the nextVal function like this: - -```Kotlin -val nextVal = myseq.nextVal() -val id = StarWarsFilms.insertAndGetId { - it[id] = nextVal - it[name] = "The Last Jedi" - it[sequelId] = 8 - it[director] = "Rian Johnson" -} -``` - -```Kotlin -val firstValue = StarWarsFilms.select(nextVal).single()[nextVal] -``` - -## Batch Insert - -Batch Insert allow mapping a list of entities into DB raws in one sql statement. It is more efficient than inserting one by one as it initiates only one statement. -Here is an example that uses a simple list: - -```kotlin -val cityNames = listOf("Paris", "Moscow", "Helsinki") -val allCitiesID = cities.batchInsert(cityNames) { name -> - this[cities.name] = name -} -``` - -Here is an example that uses a list of data class instances: - -```kotlin -data class SWFilmData(val sequelId: Int, val name: String, val director: String) - -transaction { - // ... - val films = listOf( - SWFilmData(5, "The Empire Strikes Back", "Irvin Kershner"), - SWFilmData(4, "A New Hope", "George Lucas"), - SWFilmData(7, "The Force Awakens", "JJ Abrams") - ) - - StarWarsFilms.batchInsert(films) { (id, name, director) -> - this[StarWarsFilms.sequelId] = id - this[StarWarsFilms.name] = name - this[StarWarsFilms.director] = director - } - - StarWarsFilms.selectAll().count() // 3 -} -``` - - -The `batchInsert` function will still create multiple `INSERT` statements when interacting with your database. - -You most likely want to couple this with the `rewriteBatchedInserts=true` (or `rewriteBatchedStatements=true`) option of your relevant JDBC driver, which will convert those into a single bulkInsert. - -You can find the documentation for this option for MySQL [here](https://dev.mysql.com/doc/connector-j/en/connector-j-connp-props-performance-extensions.html#cj-conn-prop_rewriteBatchedStatements) and -PostgresSQL [here](https://jdbc.postgresql.org/documentation/use/). - - -If you don't need to get the newly generated values (example: auto incremented ID), set the `shouldReturnGeneratedValues` parameter to false, this increases the -performance of batch inserts by batching them in chunks, instead of always waiting for the database to synchronize the newly inserted object state. - -If you want to check if the `rewriteBatchedInserts` + `batchInsert` is working correctly, check how to enable JDBC logging for your driver because Exposed will always -show the non-rewritten multiple inserts. You can find the documentation for how to enable logging in -PostgresSQL [here](https://jdbc.postgresql.org/documentation/logging/). - -## Insert From Select - -If you want to use the `INSERT INTO ... SELECT ` SQL clause try the function `Table.insert(Query)`: - -```kotlin -val substring = users.name.substring(1, 2) -cities.insert(users.select(substring).orderBy(users.id).limit(2)) -``` - -By default, it will try to insert into all non auto-increment `Table` columns in the order they are defined in the `Table` instance. If you want to specify columns or change the -order, provide a list of columns as the second parameter: - -```kotlin -val userCount = users.selectAll().count() -users.insert(users.select(stringParam("Foo"), Random().castTo(VarCharColumnType()).substring(1, 10)), columns = listOf(users.name, users.id)) -``` - -## Insert Or Ignore - -If supported by your specific database, `insertIgnore()` allows insert statements to be executed without throwing any -ignorable errors. This may be useful, for example, when insertion conflicts are possible: -```kotlin -StarWarsFilms.insert { - it[sequelId] = 8 // column pre-defined with a unique index - it[name] = "The Last Jedi" - it[director] = "Rian Johnson" -} -// If insert() was used, this would throw a constraint violation exception -// Instead, this new row is ignored and discarded -StarWarsFilms.insertIgnore { - it[sequelId] = 8 - it[name] = "The Rise of Skywalker" - it[director] = "JJ Abrams" -} -``` - -## Insert Or Update - -Insert or update (Upsert) is a database operation that either inserts a new row or updates an existing row if a duplicate -constraint already exists. The supported functionality of `upsert()` is dependent on the specific database being used. -For example, MySQL's `INSERT ... ON DUPLICATE KEY UPDATE` statement automatically assesses the primary key and unique indices -for a duplicate value, so using the function in Exposed would look like this: -```kotlin -// inserts a new row -StarWarsFilms.upsert { - it[sequelId] = 9 // column pre-defined with a unique index - it[name] = "The Rise of Skywalker" - it[director] = "Rian Johnson" -} -// updates existing row with the correct [director] -StarWarsFilms.upsert { - it[sequelId] = 9 - it[name] = "The Rise of Skywalker" - it[director] = "JJ Abrams" -} -``` - -If none of the optional arguments are provided to `upsert()`, and an `onUpdate()` block is omitted, the statements in the `body` block will be used for both the insert and update parts of the operation. -This means that, for example, if a table mapping has columns with default values and these columns are omitted from the `body` block, the default values will be -used for insertion as well as for the update operation. - -> If the update operation should differ from the insert operation, then `onUpdate` should be provided an argument to set -> the specific columns to update, as seen in the example below. -> -> If the update operation involves functions that should use the values that would have been inserted, then these columns -> should be marked using `insertValue()`, as seen in the example below. -{style="note"} - -Using another example, PostgreSQL allows more control over which key constraint columns to check for conflict, whether different -values should be used for an update, and whether the update statement should have a `WHERE` clause: -```kotlin -StarWarsFilms.upsert( - StarWarsFilms.sequelId, - onUpdate = { it[StarWarsFilms.sequelId] = StarWarsFilms.sequelId + 1 }, - where = { StarWarsFilms.director like stringLiteral("JJ%") } -) { - it[sequelId] = 9 - it[name] = "The Rise of Skywalker" - it[director] = "JJ Abrams" -} - -StarWarsFilms.upsert( - onUpdate = { - it[StarWarsFilms.director] = concat(insertValue(StarWarsFilms.director), stringLiteral(" || "), StarWarsFilms.director) - } -) { - it[sequelId] = 9 - it[name] = "The Rise of Skywalker" - it[director] = "Rian Johnson" -} -``` -If the update operation should be identical to the insert operation except for a few columns, -then `onUpdateExclude` should be provided an argument with the specific columns to exclude. -This parameter could also be used for the reverse case when only a small subset of columns should be updated but duplicating the insert values is tedious: -```kotlin -// on conflict, all columns EXCEPT [director] are updated with values from the lambda block -StarWarsFilms.upsert(onUpdateExclude = listOf(StarWarsFilms.director)) { - it[sequelId] = 9 - it[name] = "The Rise of Skywalker" - it[director] = "JJ Abrams" -} - -// on conflict, ONLY column [director] is updated with value from the lambda block -StarWarsFilms.upsert( - onUpdateExclude = StarWarsFilms.columns - setOf(StarWarsFilms.director) -) { - it[sequelId] = 9 - it[name] = "The Rise of Skywalker" - it[director] = "JJ Abrams" -} -``` -If a specific database supports user-defined key columns and none are provided, the table's primary key is used. If there -is no defined primary key, the first unique index is used. If there are no unique indices, each database handles this case -differently, so it is strongly advised that keys are defined to avoid unexpected results. - -> Databases that do not support a specific Insert or Update command implement the standard `MERGE INTO ... USING` statement with aliases and a derived table column list. -> These include Oracle, SQL Server, and H2 compatibility modes (except for MySQL mode). -> Any columns defined as key constraints (to be used in the `ON` clause) must be included in the statement block to avoid throwing an error. -{style="note"} - -## Replace - -SQLite, MySQL, and MariaDB (as well as the H2 compatibility modes of the latter 2 databases) support a `REPLACE` statement that acts in a similar manner -to an `INSERT OR UPDATE` statement. The only difference is that, if an insertion would violate a unique constraint, the existing row is deleted (not updated) -before the new row is inserted. - -```kotlin -object StarWarsFilms : Table() { - val sequelId: Column = integer("sequel_id").uniqueIndex() - val releaseYear: Column = integer("release_year") - val name: Column = varchar("name", 50) - val director: Column = varchar("director", 50) - val rating: Column = double("rating").default(10.0) - - override val primaryKey = PrimaryKey(sequelId, releaseYear) -} - -transaction { - // ... - // inserts a new row with default rating - StarWarsFilms.replace { - it[sequelId] = 9 - it[releaseYear] = 2019 - it[name] = "The Rise of Skywalker" - it[director] = "JJ Abrams" - } - // deletes existing row and inserts new row with set [rating] - StarWarsFilms.replace { - it[sequelId] = 9 - it[releaseYear] = 2019 - it[name] = "The Rise of Skywalker" - it[director] = "JJ Abrams" - it[rating] = 5.2 - } -} -``` - -Unlike Insert or Update, none of the supporting databases allows a `WHERE` clause. -Also, the constraints used to assess a violation are limited to the primary key and unique indexes, so there is no parameter for a custom key set. - -The values specified in the statement block will be used for the insert statement, and any omitted columns are set to their default values, if applicable. - - -In the example above, if the original row was inserted with a user-defined rating, then replace() was executed with a block that omitted the rating column, -the newly inserted row would store the default rating value. This is because the old row was completely deleted first. - - -The `REPLACE INTO ... SELECT ` SQL clause can be used by instead providing a query to `replace()`: - -```kotlin -val allRowsWithLowRating: Query = StarWarsFilms.selectAll().where { - StarWarsFilms.rating less 5.0 -} -StarWarsFilms.replace(allRowsWithLowRating) -``` - -By default, it will try to insert into all non auto-increment `Table` columns in the order they are defined in the `Table` instance. -If the columns need to be specified or the order should be changed, provide a list of columns as the second parameter: - -```kotlin -val oneYearLater = StarWarsFilms.releaseYear.plus(1) -val allRowsWithNewYear: Query = StarWarsFilms.select( - oneYearLater, StarWarsFilms.sequelId, StarWarsFilms.director, StarWarsFilms.name -) -StarWarsFilms.replace( - allRowsWithNewYear, - columns = listOf(StarWarsFilms.releaseYear, StarWarsFilms.sequelId, StarWarsFilms.director, StarWarsFilms.name) -) -``` - -## Column transformation - -Column transformations allow to define custom transformations between database column types and application's data types. -This can be particularly useful when you need to store data in one format but work with it in another format within your application. - -Consider the following example, where we define a table to store meal times and transform these times into meal types: - -```kotlin -enum class Meal { - BREAKFAST, - LUNCH, - DINNER -} - -object Meals : Table() { - val mealTime: Column = time("meal_time") - .transform( - wrap = { - when { - it.hour < 10 -> Meal.BREAKFAST - it.hour < 15 -> Meal.LUNCH - else -> Meal.DINNER - } - }, - unwrap = { - when (it) { - Meal.BREAKFAST -> LocalTime(8, 0) - Meal.LUNCH -> LocalTime(12, 0) - Meal.DINNER -> LocalTime(18, 0) - } - } - ) -} -``` - -The `transform` function is used to apply custom transformations to the `mealTime` column: - -- The `wrap` function transforms the stored `LocalTime` values into `Meal` enums. It checks the hour of the stored time and returns the corresponding meal type. -- The `unwrap` function transforms `Meal` enums back into `LocalTime` values for storage in the database. - -Transformation could be also defined as an implementation of `ColumnTransformer` interface and reused among different tables: - -```kotlin -class MealTimeTransformer : ColumnTransformer { - override fun wrap(value: LocalTime): Meal = when { - value.hour < 10 -> Meal.BREAKFAST - value.hour < 15 -> Meal.LUNCH - else -> Meal.DINNER - } - - override fun unwrap(value: Meal): LocalTime = when (value) { - Meal.BREAKFAST -> LocalTime(8, 0) - Meal.LUNCH -> LocalTime(12, 0) - Meal.DINNER -> LocalTime(18, 0) - } -} - -object Meals : Table() { - val mealTime: Column = time("meal_time").transform(MealTimeTransformer()) -} -``` - -## Custom Select Queries - -A `Query` instance, which can be instantiated by calling `selectAll()` or `select()` on a `Table` or `Join`, has many extension functions for building complex queries. -Some of these have already been mentioned above, like [where()](Deep-Dive-into-DSL.md#where-expression), [groupBy()](Deep-Dive-into-DSL.md#group-by), and [orderBy()](Deep-Dive-into-DSL.md#order-by). - -If a `SELECT` query with a special clause is required, a custom extension function can be implemented to enable its use with other standard queries. - -For example, MySQL index hints, which follow the table name in SQL, can be implemented on a `SELECT` query by using the following custom function and class: -```kotlin -fun Query.indexHint(hint: String) = IndexHintQuery(this, hint) - -class IndexHintQuery( - val source: Query, - val indexHint: String -) : Query(source.set, source.where) { - - init { - // copies any stored properties from the original query - source.copyTo(this) - } - - override fun prepareSQL(builder: QueryBuilder): String { - val originalSql = super.prepareSQL(builder) - val fromTableSql = " FROM ${transaction.identity(set.source as Table)} " - return originalSql.replace(fromTableSql, "$fromTableSql$indexHint ") - } - - override fun copy(): IndexHintQuery = IndexHintQuery(source.copy(), indexHint).also { copy -> - copyTo(copy) - } -} - -transaction { - val originalQuery = StarWarsFilms - .selectAll() - .withDistinct() - .where { StarWarsFilms.sequelId less 5 } - .groupBy(StarWarsFilms.director) - - val queryWithHint = originalQuery - .indexHint("FORCE INDEX (PRIMARY)") - .orderBy(StarWarsFilms.sequelId) -} -``` diff --git a/documentation-website/Writerside/topics/Frequently-Asked-Questions.md b/documentation-website/Writerside/topics/Frequently-Asked-Questions.md index 84a7d0df8c..a8898cf7f7 100644 --- a/documentation-website/Writerside/topics/Frequently-Asked-Questions.md +++ b/documentation-website/Writerside/topics/Frequently-Asked-Questions.md @@ -10,15 +10,15 @@ A: Yes. See [Transactions](Transactions.md#working-with-multiple-databases) ### Q: Is `Array` column type supported? -A: Yes. See [Data Types](Data-Types.md#how-to-use-array-types). +A: Yes. See [Data Types](Data-Types.topic#how-to-use-array-types). ### Q: Is `upsert` supported? -A: Yes. See [Insert Or Update](Deep-Dive-into-DSL.md#insert-or-update) +A: Yes. See [Insert Or Update](DSL-CRUD-operations.topic#insert-or-update) ### Q: Is `json` type supported? -A: Yes. See [JSON](Data-Types.md#how-to-use-json-and-jsonb-types) +A: Yes. See [JSON](Data-Types.topic#how-to-use-json-and-jsonb-types) ### Q: How to get a plain SQL query which will be executed? diff --git a/documentation-website/Writerside/topics/Getting-Started-with-Exposed.topic b/documentation-website/Writerside/topics/Getting-Started-with-Exposed.topic index 4e7256f81f..f06531db2a 100644 --- a/documentation-website/Writerside/topics/Getting-Started-with-Exposed.topic +++ b/documentation-website/Writerside/topics/Getting-Started-with-Exposed.topic @@ -503,7 +503,7 @@

    Great job! You have now implemented a simple console application that uses Exposed to fetch and modify task data from an in-memory database. Now that you’ve covered the basics, you are ready to - dive deep into the DSL API. + dive deep into the DSL API.

    diff --git a/documentation-website/Writerside/topics/Home.topic b/documentation-website/Writerside/topics/Home.topic index 470566dab4..83bc248626 100644 --- a/documentation-website/Writerside/topics/Home.topic +++ b/documentation-website/Writerside/topics/Home.topic @@ -30,8 +30,8 @@ summary="Learn how to configure Exposed in the existing project using Gradle or Maven build systems"> Adding Dependencies - - + + Querying a Database - -# Defining Tables - -This page shows what table types Exposed supports and how to define and create these tables. It also contains tips on configuring -constraints, such as `PRIMARY KEY`, `DEFAULT`, and `INDEX`. All examples use the H2 database to generate SQL. - -## Table Types - -The most primitive table type is `Table`. It is located in the **org.jetbrains.exposed.sql** package of the **exposed-core** module. -To configure a custom name for a table, which will be used in actual SQL queries, pass it to the `name` parameter of the `Table()` constructor. -Otherwise, Exposed will generate it from the full class name or the class name without the suffix 'Table', if present. - -For example, to create a simple table with an integer `id` column and a string `name` column, use any of the following options: - -Omit the `name` parameter to generate the table name from the object name: -```kotlin -object Cities : Table() { - val id = integer("id") - val name = varchar("name", 50) -} -``` -```sql -CREATE TABLE IF NOT EXISTS CITIES (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) -``` -Omit the `name` parameter to generate the table name from the object name, with any 'Table' suffix removed: -```kotlin -object CitiesTable : Table() { - val id = integer("id") - val name = varchar("name", 50) -} -``` -```sql -CREATE TABLE IF NOT EXISTS CITIES (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) -``` -Provide an argument to `name` to generate a specific table name: -```kotlin -object Cities : Table("all_cities") { - val id = integer("id") - val name = varchar("name", 50) -} -``` -```sql -CREATE TABLE IF NOT EXISTS ALL_CITIES (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) -``` -Some databases, like H2, fold unquoted identifiers to upper case. To keep table name case-sensitivity, manually quote the provided argument: -```kotlin -object Cities : Table("\"all_cities\"") { - val id = integer("id") - val name = varchar("name", 50) -} -``` -```sql -CREATE TABLE IF NOT EXISTS "all_cities" (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) -``` - -Depending on what DBMS you use, the types of columns could be different in actual SQL queries. - -### IdTable Types - -Exposed also provides the base `IdTable` class, which is inherited by `IntIdTable`, `LongIdTable` (and their unsigned variants), `UUIDTable`, and `CompositeIdTable` classes from the -**org.jetbrains.exposed.dao.id** package of the **exposed-core** module. - -These tables could be declared without the `id` column, and IDs of the appropriate type would be generated automatically when creating new table rows. -To configure a custom name for the `id` column, pass it to the `columnName` parameter of the appropriate table constructor. - -For example, the `Cities` table could instead be defined as an `IntIdTable`, which would make the `id` column both auto-incrementing and the table's primary key: -```kotlin -object Cities : IntIdTable() { - val name = varchar("name", 50) -} -``` -```sql -CREATE TABLE IF NOT EXISTS CITIES (ID INT AUTO_INCREMENT PRIMARY KEY, "name" VARCHAR(50) NOT NULL) -``` - -For more information on IdTable types, see DAO Table Types. - -## Constraints - -### Nullable - -The `NOT NULL` SQL constraint restricts the column to accept the `null` value. By default, Exposed applies this constraint to -all the columns. To allow the column to be nullable, apply the `nullable()` method to a definition of an appropriate column. - -For example, to make the population column `nullable`, use the following code: -```kotlin -// SQL: POPULATION INT NULL -val population: Column = integer("population").nullable() -``` - -### Default - -The `DEFAULT` SQL constraint provides the default value for the column. Exposed supports three methods for configuring -default values: - -* `default(defaultValue: T)` accepts a value with a type of the column. -* `defaultExpression(defaultValue: Expression)` accepts an expression. -* `clientDefault(defaultValue: () -> T)` accepts a function. - -For example, to configure the default value for the `name` column, use the following code: -```kotlin -// SQL: "NAME" VARCHAR(50) DEFAULT 'Unknown' -val name: Column = varchar("name", 50).default("Unknown") -``` - -Exposed also supports marking a column as `databaseGenerated` if the default value of the column is not known at the -time of table creation and/or if it depends on other columns. It makes it possible to omit setting a value for the -column when inserting a new record, without getting an error. The value for the column can be set by creating a TRIGGER -or with a DEFAULT clause, for example. - -For example: -```kotlin -val name: Column = varchar("name", 50).databaseGenerated() -``` - -### Index - -The `INDEX` SQL constraint makes traversing through tables quicker. Exposed supports the `index()` method. -It has six parameters, most of which are optional: - -* `val customIndexName: String? = null` is a custom name for the index, which will be used in actual SQL queries. -* `val unique: Boolean` defines whether the index is unique or not. -* `val columns: List>` defines a column set. -* `val functions: List>? = null` defines functional key parts. -* `val indexType: String? = null` is a custom type. Can be `"BTREE"` or `"HASH"`. -* `val filterCondition: (SqlExpressionBuilder.() -> Op)? = null` defines a condition used to create a partial index. - -The simplest way to create an index is to use an extension function directly on a column. For example, to apply a non-unique -`INDEX` constraint to the `name` column, use the following code: -```kotlin -val name = varchar("name", 50).index() -``` -If the parameter `customIndexName` is not set, the name of the index is determined by the table and column names. - -Also, Exposed supports complex indexes. If you have a frequent query for two columns, Exposed can perform it more efficiently. -It creates a tree from the first column with the references to the second one. For example, to create a non-unique complex -index on the `name` and `population` columns, paste the following code: -```kotlin -val indexName = index("indexName", false, *arrayOf(name, population)) -// or inside an init block within the table object -init { - index("indexName", isUnique = false, name, population) -} -``` - -Exposed also supports creating an index with a custom type. For example, to retrieve data from the `name` column faster -with a hash function for traversing, use the following code: -```kotlin -val indexName = index("indexName", false, *arrayOf(name), indexType = "HASH") -``` - -Some databases support functional key parts that index expressions instead of columns directly: -```kotlin -init { - index(functions = listOf(name.lowerCase(), address.substring(1, 5))) - uniqueIndex(columns = arrayOf(name), functions = listOf(Coalesce(address, stringLiteral("*")))) -} -``` -Operator expressions, like `plus()`, are also accepted by the `functions` parameter. - -Some databases support creating a partial index by defining a filter expression to improve querying performance. The -created index will only contain entries for the table rows that match this predicate: -```kotlin -init { - index(columns = arrayOf(name, flag)) { flag eq true } - index(columns = arrayOf(name, population)) { (name like "A%") and (population greaterEq 10) } -} -``` - -Once a table has been created, the list of its indices can be accessed using the property `Table.indices`. Table indices -are represented by the data class `Index`, so its properties can be checked in the following manner, for example: -```kotlin -Table.indices.map { it.indexName to it.createStatement().first() } -``` - - -An instance of the Index data class can be created directly using its public constructor, for the purpose of -evaluating or using create/modify/drop statements, for example. Doing so will not add the instance to an existing table's -list of indices in the way that using index() would. Also, if an instance is created with arguments provided to the -functions parameter, a functionsTable argument must also be provided. - - -### Unique - -The `UNIQUE` SQL constraint restricts duplicates within this column. Exposed supports the `uniqueIndex()` method which -creates a unique index for the column. This method is the composition of `UNIQUE` and `INDEX` constraint, the quicker -modification of `UNIQUE` constraint. - -For example, to apply `UNIQUE` and `INDEX` constraint to the `name` column, use the following code: -```kotlin -val name = varchar("name", 50).uniqueIndex() -``` - -### Primary Key - -The `PRIMARY KEY` SQL constraint applied to a column means each value in that column identifies the row. This constraint is the composition -of `NOT NULL` and `UNIQUE` constraints. To change the column set, add columns, or change the primary key name to a custom one, override this field of the table class. - -For example, to define the `name` column as the primary key, use the following code. The "Cities_name" string -will be used as the constraint name in the actual SQL query, if provided; otherwise a name will be generated based on the table's name. -```kotlin -override val primaryKey = PrimaryKey(name, name = "Cities_name") -``` -```sql -CONSTRAINT Cities_name PRIMARY KEY ("name") -``` - -It is also possible to define a primary key on a table using multiple columns: -```kotlin -override val primaryKey = PrimaryKey(id, name) -``` -```sql -CONSTRAINT pk_Cities PRIMARY KEY (ID, "name") -``` - -Except for `CompositeIdTable`, each available class in Exposed that inherits from `IdTable` has the `primaryKey` field automatically defined. -For example, the `IntIdTable` by default has an auto-incrementing integer column, `id`, which is defined as the primary key. - -An `IdTable` that requires a primary key with multiple columns can be defined using `CompositeIdTable`. -In this case, each column that is a component of the table's `id` should be identified by `entityId()`: -```kotlin -object Towns : CompositeIdTable("towns") { - val areaCode = integer("area_code").autoIncrement().entityId() - val latitude = decimal("latitude", 9, 6).entityId() - val longitude = decimal("longitude", 9, 6).entityId() - val name = varchar("name", 32) - - override val primaryKey = PrimaryKey(areaCode, latitude, longitude) -} -``` -For more information on CompositeIdTable types, see DAO Table Types. - -### Foreign Key - -The `FOREIGN KEY` SQL constraint links two tables. A foreign key is a column from one table that refers to the primary key -or columns with a unique index from another table. To configure a foreign key on a column, use `reference()` or `optReference()` -methods. The latter lets the foreign key accept a `null` value. To configure a foreign key on multiple columns, -use `foreignKey()` directly within an `init` block. - -`reference()` and `optReference()` methods have several parameters: - -`name: String` -: A name for the foreign key column, which will be used in actual SQL queries. - -`ref: Column` -: A target column from another parent table. - -`onDelete: ReferenceOption? = null` -: An action for when a linked row from a parent table will be deleted. - -`onUpdate: ReferenceOption? = null` -: An action for when a value in a referenced column will be changed. - -`fkName: String? = null` -: A name for the foreign key constraint. - -Enum class `ReferenceOption` has five values: - -`RESTRICT` -: An option that restricts changes on a referenced column, and the default option for most dialects. - -`NO_ACTION` -: The same as RESTRICT in some, but not all, databases, and the default option for Oracle and SQL Server dialects. - -`CASCADE` -: An option that allows updating or deleting the referring rows. - -`SET_NULL` -: An option that sets the referring column values to null. - -`SET_DEFAULT` -: An option that sets the referring column values to the default value. - -Consider the following `Citizens` table. This table has the `name` and `city` columns. If the `Cities` table has -configured the `name` column as the primary key, the `Citizens` table can refer to it by its `city` column, which is a foreign key. To -configure such reference and make it nullable, use the `optReference()` method: -```kotlin -object Citizens : IntIdTable() { - val name = varchar("name", 50) - val city = optReference("city", Cities.name, onDelete=ReferenceOption.CASCADE) -} -``` - -If any `Cities` row will be deleted, the appropriate `Citizens` row will be deleted too. - -If instead the `Cities` table has configured multiple columns as the primary key (for example, both `id` and `name` columns as in the above [section](#primary-key)), -the `Citizens` table can refer to it by using a table-level foreign key constraint. In this case, the `Citizens` table must have defined matching columns -to store each component value of the `Cities` table's primary key: -```kotlin -object Citizens : IntIdTable() { - val name = varchar("name", 50) - val cityId = integer("city_id") - val cityName = varchar("city_name", 50) - - init { - foreignKey(cityId, cityName, target = Cities.primaryKey) - } -} -``` - -In the above example, the order of the referencing columns in `foreignKey()` must match the order of columns defined in the target primary key. -If this order is uncertain, the foreign key can be defined with explicit column associations instead: -```kotlin -init { - foreignKey(cityId to Cities.id, cityName to Cities.name) -} -``` - -### Check - -The `CHECK` SQL constraint checks that all values in a column match some condition. Exposed supports the `check()` method. -You apply this method to a column and pass the appropriate condition to it. - -For example, to check that the `name` column contains strings that begin with a capital letter, use the following code: -```kotlin -// SQL: CONSTRAINT check_Cities_0 CHECK (REGEXP_LIKE("NAME", '^[A-Z].*', 'c'))) -val name = varchar("name", 50).check { it regexp "^[A-Z].*" } -``` - -Some databases, like older MySQL versions, may not support `CHECK` constraints. For more information, consult the relevant documentation. diff --git a/documentation-website/Writerside/topics/Table-Definition.topic b/documentation-website/Writerside/topics/Table-Definition.topic new file mode 100644 index 0000000000..a13da2530b --- /dev/null +++ b/documentation-website/Writerside/topics/Table-Definition.topic @@ -0,0 +1,440 @@ + + + + + +

    This page shows what table types Exposed supports and how to define and create these tables. It also contains + tips on configuring + constraints, such as PRIMARY KEY, DEFAULT, and INDEX. All examples use + the H2 database to generate SQL.

    + + + Core <code>Table</code> class +

    In Exposed, the Table class is the core abstraction for defining database tables. + This class provides methods to define various column types, + constraints, and other table-specific properties. +

    +

    It is located in the + org.jetbrains.exposed.sql + package of the + exposed-core + module.

    +

    + Exposed supports a variety of column types, including integer, + varchar,bool, and more. Each column is defined by calling the appropriate + method on the Table object. +

    +

    The following example defines a simple table with an integer id column and a string + name column:

    + + object Cities : Table() { + val id = integer("id") + val name = varchar("name", 50) + } + +

    + By default, Exposed will generate the table name from the full class name: +

    + + CREATE TABLE IF NOT EXISTS CITIES (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) + +

    + If the object name contains a 'Table' suffix, Exposed will omit the suffix from the generated + table name: +

    + + object CitiesTable : Table() { + val id = integer("id") + val name = varchar("name", 50) + } + + + CREATE TABLE IF NOT EXISTS CITIES (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) + + + Configure a custom table name + To configure a custom name for a table, which will be used in actual SQL queries, pass it to the name + parameter of the Table() constructor. + + object Cities : Table("all_cities") { + val id = integer("id") + val name = varchar("name", 50) + } + + + CREATE TABLE IF NOT EXISTS ALL_CITIES (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) + +

    Some databases, like H2, fold unquoted identifiers to upper case. To keep table name case-sensitivity, + manually quote the provided argument:

    + + object Cities : Table("\"all_cities\"") { + val id = integer("id") + val name = varchar("name", 50) + } + + + CREATE TABLE IF NOT EXISTS "all_cities" (ID INT NOT NULL, "name" VARCHAR(50) NOT NULL) + +

    Depending on what DBMS you use, the types of columns could be different in actual SQL queries.

    +     +     + + <code>IdTable</code> +

    Exposed also provides the base IdTable class, which is inherited by IntIdTable, + LongIdTable (and their unsigned variants), UUIDTable, and CompositeIdTable + classes from the + org.jetbrains.exposed.dao.id + package of the + exposed-core + module. +

    +

    These tables could be declared without the id column, and IDs of the appropriate type would + be generated automatically when creating new table rows. + To configure a custom name for the id column, pass it to the columnName + parameter of the appropriate table constructor.

    +

    For example, the Cities table could instead be defined as an IntIdTable, which + would make the id column both auto-incrementing and the table's primary key:

    + + + object Cities : IntIdTable() { + val name = varchar("name", 50) + } + + + + CREATE TABLE IF NOT EXISTS CITIES (ID INT AUTO_INCREMENT PRIMARY KEY, "name" VARCHAR(50) NOT NULL) + + For more information on IdTable types, see DAO + Table Types. + +     +     + + +

    The NOT NULL SQL constraint restricts the column to accept the null value. By + default, Exposed applies this constraint to + all the columns. To allow the column to be nullable, apply the nullable() method to a + definition of an appropriate column.

    +

    For example, to make the population column nullable, use the following code:

    + + + // SQL: POPULATION INT NULL + val population: Column<Int?> = integer("population").nullable() + +     + +

    The DEFAULT SQL constraint provides the default value for the column. Exposed supports three + methods for configuring + default values:

    + +
  • default(defaultValue: T) accepts a value with a type of the column.
    • +
  • defaultExpression(defaultValue: Expression<T>) accepts an expression.
    • +
  • clientDefault(defaultValue: () -> T) accepts a function.
    • +     +

    For example, to configure the default value for the name column, use the following code:

    + + + // SQL: "NAME" VARCHAR(50) DEFAULT 'Unknown' + val name: Column<String> = varchar("name", 50).default("Unknown") + +

    Exposed also supports marking a column as databaseGenerated if the default value of the + column is not known at the + time of table creation and/or if it depends on other columns. It makes it possible to omit setting a + value for the + column when inserting a new record, without getting an error. The value for the column can be set by + creating a TRIGGER + or with a DEFAULT clause, for example.

    +

    For example:

    + + + val name: Column<String> = varchar("name", 50).databaseGenerated() + +     + +

    The INDEX SQL constraint makes traversing through tables quicker. Exposed supports the + index() method. + It has six parameters, most of which are optional:

    + +
  • val customIndexName: String? = null is a custom name for the index, which will be used + in actual SQL queries. +
    • +
  • val unique: Boolean defines whether the index is unique or not.
    • +
  • val columns: List<Column<*>> defines a column set.
    • +
  • val functions: List<ExpressionWithColumnType<*>>? = null defines functional + key parts. +
    • +
  • val indexType: String? = null is a custom type. Can be "BTREE" + or "HASH". +
    • +
  • val filterCondition: (SqlExpressionBuilder.() -> Op<Boolean>)? = null defines + a condition used to create a partial index. +
    • +     +

    The simplest way to create an index is to use an extension function directly on a column. For example, to + apply a non-unique + INDEX constraint to the name column, use the following code:

    + + + val name = varchar("name", 50).index() + +

    If the parameter customIndexName is not set, the name of the index is determined by the + table and column names.

    +

    Also, Exposed supports complex indexes. If you have a frequent query for two columns, Exposed can perform + it more efficiently. + It creates a tree from the first column with the references to the second one. For example, to create a + non-unique complex + index on the name and population columns, paste the following code:

    + + + val indexName = index("indexName", false, *arrayOf(name, population)) + // or inside an init block within the table object + init { + index("indexName", isUnique = false, name, population) + } + +

    Exposed also supports creating an index with a custom type. For example, to retrieve data from the name + column faster + with a hash function for traversing, use the following code:

    + + + val indexName = index("indexName", false, *arrayOf(name), indexType = "HASH") + +

    Some databases support functional key parts that index expressions instead of columns directly:

    + + + init { + index(functions = listOf(name.lowerCase(), address.substring(1, 5))) + uniqueIndex( + columns = arrayOf(name), + functions = listOf(Coalesce(address, stringLiteral("*"))) + ) + } + +

    Operator expressions, like plus(), are also accepted by the functions + parameter.

    +

    Some databases support creating a partial index by defining a filter expression to improve querying + performance. The + created index will only contain entries for the table rows that match this predicate:

    + + + init { + index(columns = arrayOf(name, flag)) { flag eq true } + index( + columns = arrayOf( + name, + population + ) + ) { (name like "A%") and (population greaterEq 10) } + } + +

    Once a table has been created, the list of its indices can be accessed using the property Table.indices. + Table indices + are represented by the data class Index, so its properties can be checked in the following + manner, for example:

    + + + Table.indices.map { it.indexName to it.createStatement().first() } + + + An instance of the Index data class can be created directly using its public constructor, + for the purpose of + evaluating or using create/modify/drop statements, for example. Doing so will not add the instance to an + existing table's + list of indices in the way that using index() would. Also, if an instance is created with + arguments provided to the + functions parameter, a functionsTable argument must also be provided. + +     + +

    The UNIQUE SQL constraint restricts duplicates within this column. Exposed supports the + uniqueIndex() method which + creates a unique index for the column. This method is the composition of UNIQUE and INDEX + constraint, the quicker + modification of UNIQUE constraint.

    +

    For example, to apply UNIQUE and INDEX constraint to the name + column, use the following code:

    + + + val name = varchar("name", 50).uniqueIndex() + +     + +

    The PRIMARY KEY SQL constraint applied to a column means each value in that column + identifies the row. This constraint is the composition + of NOT NULL and UNIQUE constraints. To change the column set, add columns, or + change the primary key name to a custom one, override this field of the table class.

    +

    For example, to define the name column as the primary key, use the following code. The + "Cities_name" string + will be used as the constraint name in the actual SQL query, if provided; otherwise a name will be + generated based on the table's name.

    + + + override val primaryKey = PrimaryKey(name, name = "Cities_name") + + + + CONSTRAINT Cities_name PRIMARY KEY ("name") + +

    It is also possible to define a primary key on a table using multiple columns:

    + + + override val primaryKey = PrimaryKey(id, name) + + + + CONSTRAINT pk_Cities PRIMARY KEY (ID, "name") + +

    Except for CompositeIdTable, each available class in Exposed that inherits from IdTable + has the primaryKey field automatically defined. + For example, the IntIdTable by default has an auto-incrementing integer column, + id, which is defined as the primary key.

    +

    An IdTable that requires a primary key with multiple columns can be defined using CompositeIdTable. + In this case, each column that is a component of the table's id should be identified by + entityId():

    + + + object Towns : CompositeIdTable("towns") { + val areaCode = integer("area_code").autoIncrement().entityId() + val latitude = decimal("latitude", 9, 6).entityId() + val longitude = decimal("longitude", 9, 6).entityId() + val name = varchar("name", 32) + + override val primaryKey = PrimaryKey(areaCode, latitude, longitude) + } + + For more information on CompositeIdTable types, see DAO Table Types. + +     + +

    The FOREIGN KEY SQL constraint links two tables. A foreign key is a column from one table + that refers to the primary key + or columns with a unique index from another table. To configure a foreign key on a column, use reference() + or optReference() + methods. The latter lets the foreign key accept a null value. To configure a foreign key on + multiple columns, + use foreignKey() directly within an init block.

    +

    reference() and optReference() methods have several parameters:

    + + + <code>name: String</code> +

    A name for the foreign key column, which will be used in actual SQL queries.

    +     + + <code>ref: Column<T></code> +

    A target column from another parent table.

    +     + + <code>onDelete: ReferenceOption? = null</code> +

    An action for when a linked row from a parent table will be deleted.

    +     + + <code>onUpdate: ReferenceOption? = null</code> +

    An action for when a value in a referenced column will be changed.

    +     + + <code>fkName: String? = null</code> +

    A name for the foreign key constraint.

    +     +     +

    Enum class ReferenceOption has five values:

    + + + <code>RESTRICT</code> +

    An option that restricts changes on a referenced column, and the default option for most + dialects.

    +     + + <code>NO_ACTION</code> +

    The same as RESTRICT in some, but not all, databases, and the default option for Oracle and SQL + Server dialects.

    +     + + <code>CASCADE</code> +

    An option that allows updating or deleting the referring rows.

    +     + + <code>SET_NULL</code> +

    An option that sets the referring column values to null.

    +     + + <code>SET_DEFAULT</code> +

    An option that sets the referring column values to the default value.

    +     +     +

    Consider the following Citizens table. This table has the name and + city columns. If the Cities table has + configured the name column as the primary key, the Citizens table can refer to + it by its city column, which is a foreign key. To + configure such reference and make it nullable, use the optReference() method:

    + + + object Citizens : IntIdTable() { + val name = varchar("name", 50) + val city = optReference("city", Cities.name, onDelete = ReferenceOption.CASCADE) + } + +

    If any Cities row will be deleted, the appropriate Citizens row will be deleted + too.

    +

    If instead the Cities table has configured multiple columns as the primary key (for example, + both id and name columns as in the above section), + the Citizens table can refer to it by using a table-level foreign key constraint. In this + case, the Citizens table must have defined matching columns + to store each component value of the Cities table's primary key:

    + + + object Citizens : IntIdTable() { + val name = varchar("name", 50) + val cityId = integer("city_id") + val cityName = varchar("city_name", 50) + + init { + foreignKey(cityId, cityName, target = Cities.primaryKey) + } + } + +

    In the above example, the order of the referencing columns in foreignKey() must match the + order of columns defined in the target primary key. + If this order is uncertain, the foreign key can be defined with explicit column associations instead: +

    + + + init { + foreignKey(cityId to Cities.id, cityName to Cities.name) + } + +     + +

    The CHECK SQL constraint checks that all values in a column match some condition. Exposed + supports the check() method. + You apply this method to a column and pass the appropriate condition to it.

    +

    For example, to check that the name column contains strings that begin with a capital + letter, use the following code:

    + + + // SQL: CONSTRAINT check_Cities_0 CHECK (REGEXP_LIKE("NAME", '^[A-Z].*', 'c'))) + val name = varchar("name", 50).check { it regexp "^[A-Z].*" } + +

    Some databases, like older MySQL versions, may not support CHECK constraints. For more + information, consult the relevant documentation.

    +     +     + +

    + To create a table within a database, you need to use the SchemaUtils.create() + method within a transaction: +

    + + transaction { + SchemaUtils.create(StarWarsFilms) + //... + } + +

    + This will generate the SQL necessary to create the table based on your definition. +

    +     + +     diff --git a/documentation-website/Writerside/topics/Working-with-Schema.topic b/documentation-website/Writerside/topics/Working-with-Schema.topic new file mode 100644 index 0000000000..dcf83ebf56 --- /dev/null +++ b/documentation-website/Writerside/topics/Working-with-Schema.topic @@ -0,0 +1,48 @@ + + + +

    + A database schema defines how data is organized in a relational database. With Exposed, you can create and + drop a new or an existing schema. +

    + +

    + To define a schema in Exposed, use the Schema class: +

    + + val schema = Schema("my_schema") // my_schema is the schema name. + +

    Additionally, you can specify the schema owner by passing an authorization argument (some databases require the explicit owner) :

    + + + val schema = Schema("my_schema", authorization = "owner") + +     + +

    + To create a new schema, use the createSchema() method provided by SchemaUtils: +

    + + SchemaUtils.createSchema(schema) + +     + +

    If you have many schemas, and you want to set a default one, you can use the setSchema() + method from SchemaUtils: +

    + + SchemaUtils.setSchema(schema) + +     + +

    + To drop a schema, use the dropSchema() method provided by SchemaUtils: +

    + + SchemaUtils.dropSchema(schema) + +     +     diff --git a/documentation-website/Writerside/topics/Working-with-Sequence.topic b/documentation-website/Writerside/topics/Working-with-Sequence.topic new file mode 100644 index 0000000000..848ca2644d --- /dev/null +++ b/documentation-website/Writerside/topics/Working-with-Sequence.topic @@ -0,0 +1,58 @@ + + + + +

    A sequence is a database object that automatically generates integer values in sequential order. It is particularly + useful in generating unique identifiers and primary keys.

    + + To define a sequence in Exposed, use the Sequence class: + + val myseq = Sequence("my_sequence") // my_sequence is the sequence name. + +

    Several parameters can be specified to control the properties of the sequence:

    + + + private val myseq = Sequence( + name = "my_sequence", + startWith = 4, + incrementBy = 2, + minValue = 1, + maxValue = 10, + cycle = true, + cache = 20 + ) + +     + + To create a sequence, use the createSequence() method provided by SchemaUtils: + + SchemaUtils.createSequence(myseq) + + + + To drop a sequence, use the dropSequence() method provided by SchemaUtils: + + SchemaUtils.dropSequence(myseq) + + + +

    You can access the next value in a sequence by using the nextIntVal() function:

    + + + val nextVal = myseq.nextIntVal() + val id = StarWarsFilms.insertAndGetId { + it[id] = nextVal + it[name] = "The Last Jedi" + it[sequelId] = 8 + it[director] = "Rian Johnson" + } + + + + val firstValue = StarWarsFilms.select(nextVal).single()[nextVal] + +     +