feat!: EXPOSED-359 Add support for multidimensional arrays (#2250)

* feat!: EXPOSED-359 Add support for multidimensional arrays

documentation-website/Writerside/topics/Breaking-Changes.md

@@ -8,6 +8,9 @@
 * In Oracle and H2 Oracle, the `uinteger()` column now maps to data type `NUMBER(10)` instead of `NUMBER(13)`.
 * In Oracle and H2 Oracle, the `integer()` column now maps to data type `NUMBER(10)` and `INTEGER` respectively, instead of `NUMBER(12)`.
   In Oracle and SQLite, using the integer column in a table now also creates a CHECK constraint to ensure that no out-of-range values are inserted.
+* `ArrayColumnType` now supports multidimensional arrays and includes an additional generic parameter.
+  If it was previously used for one-dimensional arrays with the parameter `T` like `ArrayColumnType<T>`,
+  it should now be defined as `ArrayColumnType<T, List<T>>`. For instance, `ArrayColumnType<Int>` should now be `ArrayColumnType<Int, List<Int>>`.
 
 ## 0.55.0
 * The `DeleteStatement` property `table` is now deprecated in favor of `targetsSet`, which holds a `ColumnSet` that may be a `Table` or `Join`.

documentation-website/Writerside/topics/Data-Types.topic

@@ -252,31 +252,52 @@
         </chapter>
     </chapter>
     <chapter title="How to use Array types" id="how-to-use-array-types">
-        <p>PostgreSQL and H2 databases support the explicit ARRAY data type.</p>
-        <p>Exposed currently only supports columns defined as one-dimensional arrays, with the stored contents being any
-            out-of-the-box or custom data type.
-            If the contents are of a type with a supported <code>ColumnType</code> in the <code>exposed-core</code>
-            module, the column can be simply defined with that type:</p>
+        <p>PostgreSQL and H2 databases support the explicit ARRAY data type,
+            with multi-dimensional arrays being supported by PostgreSQL.</p>
+
+        <p>Exposed allows defining columns as arrays, with the stored contents being any out-of-the-box or custom data type.
+            If the contents are of a type with a supported <code>ColumnType</code> in the <code>exposed-core</code> module,
+            the column can be simply defined with that type:</p>
 
         <code-block lang="kotlin">
             object Teams : Table(&quot;teams&quot;) {
+                // Single-dimensional arrays
                 val memberIds = array&lt;UUID&gt;(&quot;member_ids&quot;)
                 val memberNames = array&lt;String&gt;(&quot;member_names&quot;)
                 val budgets = array&lt;Double&gt;(&quot;budgets&quot;)
+
+                // Multi-dimensional arrays
+                val nestedMemberIds = array&lt;UUID, List&lt;List&lt;UUID&gt;&gt;&gt;(
+                    &quot;nested_member_ids&quot;, dimensions = 2
+                )
+                val hierarchicalMemberNames = array&lt;String, List&lt;List&lt;List&lt;String&gt;&gt;&gt;&gt;(
+                    &quot;hierarchical_member_names&quot;, dimensions = 3
+                )
             }
         </code-block>
+
         <p>If more control is needed over the base content type, or if the latter is user-defined or from a non-core
             module, the explicit type should be provided to the function:</p>
 
         <code-block lang="kotlin">
             object Teams : Table(&quot;teams&quot;) {
-                val memberIds = array&lt;UUID&gt;(&quot;member_ids&quot;)
+                // Single-dimensional arrays
                 val memberNames = array&lt;String&gt;(&quot;member_names&quot;, VarCharColumnType(colLength = 32))
                 val deadlines = array&lt;LocalDate&gt;(&quot;deadlines&quot;, KotlinLocalDateColumnType()).nullable()
-                val budgets = array&lt;Double&gt;(&quot;budgets&quot;)
                 val expenses = array&lt;Double?&gt;(&quot;expenses&quot;, DoubleColumnType()).default(emptyList())
+
+                // Multi-dimensional arrays
+                val nestedMemberIds = array&lt;UUID, List&lt;List&lt;UUID&gt;&gt;&gt;(
+                    &quot;nested_member_ids&quot;, dimensions = 2
+                )
+                val hierarchicalMemberNames = array&lt;String, List&lt;List&lt;List&lt;String&gt;&gt;&gt;&gt;(
+                    &quot;hierarchical_member_names&quot;,
+                    VarCharColumnType(colLength = 32),
+                    dimensions = 3
+                )
             }
         </code-block>
+
         <p>This will prevent an exception being thrown if Exposed cannot find an associated column mapping for the
             defined type.
             Null array contents are allowed, and the explicit column type should be provided for these columns as
@@ -285,9 +306,16 @@
 
         <code-block lang="kotlin">
             Teams.insert {
+                // Single-dimensional arrays
                 it[memberIds] = List(5) { UUID.randomUUID() }
                 it[memberNames] = List(5) { i -&gt; &quot;Member ${'A' + i}&quot; }
                 it[budgets] = listOf(9999.0)
+
+                // Multi-dimensional arrays
+                it[nestedMemberIds] = List(5) { List(5) { UUID.randomUUID() } }
+                it[hierarchicalMemberNames] = List(3) { List(3) { List(3) {
+                        i -> "Member ${'A' + i}"
+                } } }
             }
         </code-block>
         <chapter title="Array Functions" id="array-functions">
@@ -300,6 +328,14 @@
                     .select(firstMember)
                     .where { Teams.expenses[1] greater Teams.budgets[1] }
             </code-block>
+
+            <p>This also applies to multidimensional arrays:</p>
+            <code-block lang="kotlin">
+                Teams
+                    .selectAll()
+                    .where { Teams.hierarchicalMemberNames[1][1] eq "Mr. Smith" }
+            </code-block>
+
             <note>
                 Both PostgreSQL and H2 use a one-based indexing convention, so the first element is retrieved by using
                 index 1.
@@ -310,6 +346,11 @@
             <code-block lang="kotlin">
                 Teams.select(Teams.deadlines.slice(1, 3))
             </code-block>
+
+            <p>In the case of multidimensional arrays, the <code>slice()</code> calls can be nested:</p>
+            <code-block lang="kotlin">
+                Teams.select(Teams.hierarchicalMemberNames.slice(1, 2).slice(3, 4))
+            </code-block>
             <p>Both arguments for these bounds are optional if using PostgreSQL.</p>
             <p>An array column can also be used as an argument for the <code>ANY</code> and <code>ALL</code> SQL
                 operators, either by providing the entire column or a new array expression via <code>slice()</code>:</p>

exposed-core/api/exposed-core.api

@@ -205,9 +205,12 @@ public final class org/jetbrains/exposed/sql/AndOp : org/jetbrains/exposed/sql/C
 public final class org/jetbrains/exposed/sql/ArrayColumnType : org/jetbrains/exposed/sql/ColumnType {
 	public fun <init> (Lorg/jetbrains/exposed/sql/ColumnType;Ljava/lang/Integer;)V
 	public synthetic fun <init> (Lorg/jetbrains/exposed/sql/ColumnType;Ljava/lang/Integer;ILkotlin/jvm/internal/DefaultConstructorMarker;)V
+	public fun <init> (Lorg/jetbrains/exposed/sql/ColumnType;Ljava/util/List;I)V
+	public synthetic fun <init> (Lorg/jetbrains/exposed/sql/ColumnType;Ljava/util/List;IILkotlin/jvm/internal/DefaultConstructorMarker;)V
 	public final fun getDelegate ()Lorg/jetbrains/exposed/sql/ColumnType;
 	public final fun getDelegateType ()Ljava/lang/String;
-	public final fun getMaximumCardinality ()Ljava/lang/Integer;
+	public final fun getDimensions ()I
+	public final fun getMaximumCardinality ()Ljava/util/List;
 	public synthetic fun nonNullValueAsDefaultString (Ljava/lang/Object;)Ljava/lang/String;
 	public fun nonNullValueAsDefaultString (Ljava/util/List;)Ljava/lang/String;
 	public synthetic fun nonNullValueToString (Ljava/lang/Object;)Ljava/lang/String;
@@ -219,8 +222,6 @@ public final class org/jetbrains/exposed/sql/ArrayColumnType : org/jetbrains/exp
 	public fun sqlType ()Ljava/lang/String;
 	public synthetic fun valueFromDB (Ljava/lang/Object;)Ljava/lang/Object;
 	public fun valueFromDB (Ljava/lang/Object;)Ljava/util/List;
-	public synthetic fun valueToString (Ljava/lang/Object;)Ljava/lang/String;
-	public fun valueToString (Ljava/util/List;)Ljava/lang/String;
 }
 
 public final class org/jetbrains/exposed/sql/AutoIncColumnType : org/jetbrains/exposed/sql/IColumnType {
@@ -2467,7 +2468,9 @@ public class org/jetbrains/exposed/sql/Table : org/jetbrains/exposed/sql/ColumnS
 	public fun <init> (Ljava/lang/String;)V
 	public synthetic fun <init> (Ljava/lang/String;ILkotlin/jvm/internal/DefaultConstructorMarker;)V
 	public final fun array (Ljava/lang/String;Lorg/jetbrains/exposed/sql/ColumnType;Ljava/lang/Integer;)Lorg/jetbrains/exposed/sql/Column;
+	public final fun array (Lorg/jetbrains/exposed/sql/Table;Ljava/lang/String;Lorg/jetbrains/exposed/sql/ColumnType;Ljava/util/List;I)Lorg/jetbrains/exposed/sql/Column;
 	public static synthetic fun array$default (Lorg/jetbrains/exposed/sql/Table;Ljava/lang/String;Lorg/jetbrains/exposed/sql/ColumnType;Ljava/lang/Integer;ILjava/lang/Object;)Lorg/jetbrains/exposed/sql/Column;
+	public static synthetic fun array$default (Lorg/jetbrains/exposed/sql/Table;Lorg/jetbrains/exposed/sql/Table;Ljava/lang/String;Lorg/jetbrains/exposed/sql/ColumnType;Ljava/util/List;IILjava/lang/Object;)Lorg/jetbrains/exposed/sql/Column;
 	public final fun autoGenerate (Lorg/jetbrains/exposed/sql/Column;)Lorg/jetbrains/exposed/sql/Column;
 	public final fun autoIncrement (Lorg/jetbrains/exposed/sql/Column;Ljava/lang/String;)Lorg/jetbrains/exposed/sql/Column;
 	public final fun autoIncrement (Lorg/jetbrains/exposed/sql/Column;Lorg/jetbrains/exposed/sql/Sequence;)Lorg/jetbrains/exposed/sql/Column;

exposed-core/src/main/kotlin/org/jetbrains/exposed/sql/ColumnType.kt

@@ -1233,47 +1233,79 @@ class CustomEnumerationColumnType<T : Enum<T>>(
 // Array columns
 
 /**
- * Array column for storing a collection of elements.
+ * Multi-dimensional array column type for storing a collection of nested elements.
+ *
+ * @property delegate The base column type associated with this array column's individual elements.
+ * @property dimensions The number of dimensions of the multi-dimensional array.
+ * @property maximumCardinality The maximum cardinality (number of allowed elements) for each dimension of the array.
+ *
+ * **Note:** The maximum cardinality is considered for each dimension, but it is ignored by the PostgreSQL database.
  */
-class ArrayColumnType<E>(
-    /** Returns the base column type of this array column's individual elements. */
-    val delegate: ColumnType<E & Any>,
-    /** Returns the maximum amount of allowed elements in this array column. */
-    val maximumCardinality: Int? = null
-) : ColumnType<List<E>>() {
+class ArrayColumnType<T, R : List<Any?>>(
+    val delegate: ColumnType<T & Any>,
+    val maximumCardinality: List<Int>? = null,
+    val dimensions: Int = 1
+) : ColumnType<R>() {
+    /**
+     * Constructor with maximum cardinality for a single dimension.
+     *
+     * @param delegate The base column type associated with this array column's individual elements.
+     * @param maximumCardinality The maximum cardinality (number of allowed elements) for the array.
+     */
+    constructor(delegate: ColumnType<T & Any>, maximumCardinality: Int? = null) : this(delegate, maximumCardinality?.let { listOf(it) })
+
+    /**
+     * The SQL type definition of the delegate column type without any potential array dimensions.
+     */
+    val delegateType: String
+        get() = delegate.sqlType().substringBefore('(')
+
     override fun sqlType(): String = buildString {
+        if (maximumCardinality != null) {
+            require(maximumCardinality.size == dimensions) {
+                "The size of cardinality list must be equal to the amount of array dimensions. " +
+                    "Dimensions: $dimensions, cardinality size: ${maximumCardinality.size}"
+            }
+        }
         append(delegate.sqlType())
         when {
-            currentDialect is H2Dialect -> append(" ARRAY", maximumCardinality?.let { "[$it]" } ?: "")
-            else -> append("[", maximumCardinality?.toString() ?: "", "]")
+            currentDialect is H2Dialect -> {
+                require(dimensions == 1) {
+                    "H2 does not support multidimensional arrays. " +
+                        "`dimensions` parameter for H2 database must be 1"
+                }
+                append(" ARRAY", maximumCardinality?.let { "[${it.first()}]" } ?: "")
+            }
+
+            else -> append(maximumCardinality?.let { cardinality -> cardinality.joinToString("") { "[$it]" } } ?: "[]".repeat(dimensions))
         }
     }
 
-    /** The base SQL type of this array column's individual elements without extra column identifiers. */
-    val delegateType: String
-        get() = delegate.sqlType().substringBefore('(')
-
-    @Suppress("UNCHECKED_CAST")
-    override fun valueFromDB(value: Any): List<E> = when {
-        value is java.sql.Array -> (value.array as Array<*>).map { e -> e?.let { delegate.valueFromDB(it) } as E }
-        else -> value as? List<E> ?: error("Unexpected value $value of type ${value::class.qualifiedName}")
+    override fun notNullValueToDB(value: R): Any {
+        return recursiveNotNullValueToDB(value, dimensions)
     }
 
-    override fun notNullValueToDB(value: List<E>): Any = value.map { e -> e?.let { delegate.notNullValueToDB(it) } }.toTypedArray()
-
-    override fun valueToString(value: List<E>?): String = if (value != null) nonNullValueToString(value) else super.valueToString(null)
+    private fun recursiveNotNullValueToDB(value: Any, level: Int): Array<Any?> = when {
+        level > 1 -> (value as List<Any>).map { recursiveNotNullValueToDB(it, level - 1) }.toTypedArray()
+        else -> (value as List<T>).map { it?.let { delegate.notNullValueToDB(it) } }.toTypedArray()
+    }
 
-    override fun nonNullValueToString(value: List<E>): String {
-        val prefix = if (currentDialect is H2Dialect) "ARRAY [" else "ARRAY["
-        return value.joinToString(",", prefix, "]") { delegate.valueToString(it) }
+    @Suppress("UNCHECKED_CAST")
+    override fun valueFromDB(value: Any): R? {
+        return when {
+            value is Array<*> -> recursiveValueFromDB(value, dimensions) as R?
+            else -> value as R?
+        }
     }
 
-    override fun nonNullValueAsDefaultString(value: List<E>): String {
-        val prefix = if (currentDialect is H2Dialect) "ARRAY [" else "ARRAY["
-        return value.joinToString(",", prefix, "]") { delegate.valueAsDefaultString(it) }
+    private fun recursiveValueFromDB(value: Any?, level: Int): List<Any?> = when {
+        level > 1 -> (value as Array<Any?>).map { recursiveValueFromDB(it, level - 1) }
+        else -> (value as Array<Any?>).map { it?.let { delegate.valueFromDB(it) } }
     }
 
-    override fun readObject(rs: ResultSet, index: Int): Any? = rs.getArray(index)
+    override fun readObject(rs: ResultSet, index: Int): Any? {
+        return rs.getArray(index)?.array
+    }
 
     override fun setParameter(stmt: PreparedStatementApi, index: Int, value: Any?) {
         when {
@@ -1284,6 +1316,31 @@ class ArrayColumnType<E>(
             else -> super.setParameter(stmt, index, value)
         }
     }
+
+    override fun nonNullValueToString(value: R): String {
+        return arrayLiteralPrefix() + recursiveNonNullValueToString(value, dimensions)
+    }
+
+    private fun recursiveNonNullValueToString(value: Any?, level: Int): String = when {
+        level > 1 -> (value as List<Any?>).joinToString(",", "[", "]") { recursiveNonNullValueToString(it, level - 1) }
+        else -> (value as List<T & Any>).joinToString(",", "[", "]") { delegate.nonNullValueToString(it) }
+    }
+
+    override fun nonNullValueAsDefaultString(value: R): String {
+        return arrayLiteralPrefix() + recursiveNonNullValueAsDefaultString(value, dimensions)
+    }
+
+    private fun recursiveNonNullValueAsDefaultString(value: Any?, level: Int): String = when {
+        level > 1 -> (value as List<Any?>).joinToString(",", "[", "]") { recursiveNonNullValueAsDefaultString(it, level - 1) }
+        else -> (value as List<T & Any>).joinToString(",", "[", "]") { delegate.nonNullValueAsDefaultString(it) }
+    }
+
+    private fun arrayLiteralPrefix(): String {
+        return when {
+            currentDialect is H2Dialect -> "ARRAY "
+            else -> "ARRAY"
+        }
+    }
 }
 
 private fun isArrayOfByteArrays(value: Array<*>) =

exposed-core/src/main/kotlin/org/jetbrains/exposed/sql/Op.kt

@@ -687,9 +687,24 @@ fun decimalLiteral(value: BigDecimal): LiteralOp<BigDecimal> = LiteralOp(Decimal
  *
  * @throws IllegalStateException If no column type mapping is found and a [delegateType] is not provided.
  */
-inline fun <reified T : Any> arrayLiteral(value: List<T>, delegateType: ColumnType<T>? = null): LiteralOp<List<T>> {
+inline fun <reified T : Any> arrayLiteral(value: List<T>, delegateType: ColumnType<T>? = null): LiteralOp<List<T>> =
+    arrayLiteral(value, 1, delegateType)
+
+/**
+ * Returns the specified [value] as an array literal, with elements parsed by the [delegateType] if provided.
+ *
+ * **Note** If [delegateType] is left `null`, the associated column type will be resolved according to the
+ * internal mapping of the element's type in [resolveColumnType].
+ *
+ * **Note:** Because arrays can have varying dimensions, you must specify the type of elements
+ * and the number of dimensions when using array literals.
+ * For example, use `arrayLiteral<Int, List<List<Int>>>(list, dimensions = 2)`.
+ *
+ * @throws IllegalStateException If no column type mapping is found and a [delegateType] is not provided.
+ */
+inline fun <reified T : Any, R : List<Any>> arrayLiteral(value: R, dimensions: Int, delegateType: ColumnType<T>? = null): LiteralOp<R> {
     @OptIn(InternalApi::class)
-    return LiteralOp(ArrayColumnType(delegateType ?: resolveColumnType(T::class)), value)
+    return LiteralOp(ArrayColumnType(delegateType ?: resolveColumnType(T::class), dimensions = dimensions), value)
 }
 
 // Query Parameters
@@ -776,9 +791,24 @@ fun blobParam(value: ExposedBlob, useObjectIdentifier: Boolean = false): Express
  *
  * @throws IllegalStateException If no column type mapping is found and a [delegateType] is not provided.
  */
-inline fun <reified T : Any> arrayParam(value: List<T>, delegateType: ColumnType<T>? = null): Expression<List<T>> {
+inline fun <reified T : Any> arrayParam(value: List<T>, delegateType: ColumnType<T>? = null): Expression<List<T>> =
+    arrayParam(value, 1, delegateType)
+
+/**
+ * Returns the specified [value] as an array query parameter, with elements parsed by the [delegateType] if provided.
+ *
+ * **Note** If [delegateType] is left `null`, the associated column type will be resolved according to the
+ * internal mapping of the element's type in [resolveColumnType].
+ *
+ * **Note:** Because arrays can have varying dimensions, you must specify the type of elements
+ * and the number of dimensions when using array literals.
+ * For example, use `arrayParam<Int, List<List<Int>>>(list, dimensions = 2)`.
+ *
+ * @throws IllegalStateException If no column type mapping is found and a [delegateType] is not provided.
+ */
+inline fun <reified T : Any, R : List<Any>> arrayParam(value: R, dimensions: Int, delegateType: ColumnType<T>? = null): Expression<R> {
     @OptIn(InternalApi::class)
-    return QueryParameter(value, ArrayColumnType(delegateType ?: resolveColumnType(T::class)))
+    return QueryParameter(value, ArrayColumnType(delegateType ?: resolveColumnType(T::class), dimensions = dimensions))
 }
 
 // Misc.

exposed-core/src/main/kotlin/org/jetbrains/exposed/sql/SQLExpressionBuilder.kt

@@ -198,8 +198,12 @@ fun <E, T : List<E>?> allFrom(expression: Expression<T>): Op<E> = AllAnyFromExpr
  *
  * @sample org.jetbrains.exposed.sql.tests.shared.types.ArrayColumnTypeTests.testSelectUsingArrayGet
  */
-infix operator fun <E, T : List<E>?> ExpressionWithColumnType<T>.get(index: Int): ArrayGet<E, T> =
-    ArrayGet(this, index, (this.columnType as ArrayColumnType<E>).delegate)
+infix operator fun <E, T : List<E>?> ExpressionWithColumnType<T>.get(index: Int): ArrayGet<E, T> {
+    return when (this) {
+        is ArrayGet<*, *> -> ArrayGet(this as Expression<T>, index, this.columnType as IColumnType<E & Any>) as ArrayGet<E, T>
+        else -> ArrayGet(this, index, (this.columnType as ArrayColumnType<E, List<E>>).delegate)
+    }
+}
 
 /**
  * Returns a subarray of elements stored from between [lower] and [upper] bounds (inclusive),