fix drizzle-kit push crash on compound indexes with expression members

[jaredp]

fix #3745
See my comment #3745 (comment)

drizzle-kit push crashes with confusing Zod error Expected string, received null when making incremental updates to DBs that have tables like

// approx translation of original bug reporter's example
export const translations = pgTable(
  "translation",
  {
    source_language: text("source_language"),
    source_text: text("source_text"),
    target_language: text("target_language"),
    target_text: text("target_text"),
  },
  (table) => ({
    tnsltn: index("tnsltn").on(
      table.source_language,
      sql<string>`(md5((${table.source_text})::text))`,
    ),
  }),
);

where there's an index like (col, expression(col)) such that one of the index components is expression(col), ie. not just a plain column.

The crash will not happen when the translations table is first added to the schema and drizzle-kit pushed. The crash will only happen on subsequent drizzle-kit pushes, when translation's index is already in the db.

This happens because drizzle-kit push first reads the existing schema of the db, diffs it against the schema in typescript, and makes the updates necessary. The bug is in reading from the db. If drizzle-kit encounters a compound index with an expression (non-column) component, the db schema introspection code returns null for that component.

Later, the schema introspected out of the running db is passed to Zod for validation, which is surprised at a null where it expects a sql string defining the index's nth component.

The inappropriate null is constructed in fromDatabase, where:

The dbIndexes it pulls from postgres has a column_name=NULL for the row with indexname='tnsltn', index_order=2, where it ought to have column_name='md5((source_text)::text)'.

I confirmed this in my project, which has a table with similar index to that of the original bug reporter. I worked with Claude to generate this new SQL query. I confirmed that this changed SQL query produced by Claude works for my project.

However, I'm personally not great at SQL, so do very much take this with a grain of salt. I can't meaningfully read the original query, nor the one Claude wrote.

Hopefully someone with better SQL can read and give feedback!

[Laci556]

also fixes #3062

[Laci556]

Are there any cases when a.attname and the returned value from pg_get_indexref are different for simple columns? pg_get_indexref returns the same values that the index was created with, couldn't this be simplified to just pg_get_indexdef(i.indexrelid, k.i, true) as column_name?

[jaredp]

Possibly— I have no idea!

This SQL was generated by Claude and seems to work for my use case. I can happily debug through the Typescript but my knowledge of PG introspection/internals is weak.

I'm hoping a reviewer with knowledge of pg details can make a proper assessment of the Claude-generated SQL changes— entirely possible your simplification is valid.

for simple columns

this is a fix for compound indexes where at least one component is an expression (ie not a simple column). Not sure if that affects your answer or not.

[Laci556]

By "simple columns" I meant references to columns (without expressions), such as source_language in your example. Both the original and your refactored solution work for those cases, I was just wondering if we could make it cleaner and easier to read without the conditional and array indexing 😄 Other than that, it's essentially the same patch as I made on our project and hadn't had any issues so far, hope a maintainer can review it soon!

[Mardoxx]

Related:

SQL generated on introspect:
CREATE INDEX "tnsltn" ON "translation" USING btree (source_language text_ops,md5(source_text) text_ops);

But on push it wants to do:
CREATE INDEX "tnsltn" ON "translation" USING btree ("source_language",(md5(("source_text")::text)));

I think there's a bug in how pg-core dumps the indices?