Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More information in /schema about column types. #385

Open
divyenduz opened this issue Sep 12, 2024 · 4 comments
Open

More information in /schema about column types. #385

divyenduz opened this issue Sep 12, 2024 · 4 comments
Labels
good first issue Good for newcomers

Comments

@divyenduz
Copy link

divyenduz commented Sep 12, 2024

To better support enums in the frontend, we need some more information in the /schema API, here is an example, this column (Field.usage) is an enum but we have no way of telling it from, say, a custom domain type or custom types from extensions. Eventually, we want to make all types editable from the UI. So, we need the /schema API to return that this is an enum and its possible values for enum's case. This will allow us to build features like dropdown select for enum and/or client side validation for enum value.

Current output:

{
  "schema": {
    "name": "",
    "display_name": "",
    "tables": {
      "Field": {
        "oid": "6526469",
        "name": "Field",
        "comment": "",
        "columns": {
          "usage": {
            "name": "usage",
            "type": "bb_mmco0u98b56618sjl6glp6csh8_d5mrcj.\"FieldUsage\"",
            "default": null,
            "nullable": false,
            "unique": false,
            "comment": ""
          }
        },
        "indexes": {
          "Field_pkey": {
            "name": "Field_pkey",
            "unique": true,
            "columns": ["id"]
          }
        },
        "primaryKey": ["id"],
        "foreignKeys": {},
        "checkConstraints": {},
        "uniqueConstraints": {},
        "xataCompatible": false
      }
    }
  }
}

Potential future output

{
  "schema": {
    "name": "",
    "display_name": "",
    "tables": {
      "Field": {
        "oid": "6526469",
        "name": "Field",
        "comment": "",
        "columns": {
          "usage": {
            "name": "usage",
            "type": "bb_mmco0u98b56618sjl6glp6csh8_d5mrcj.\"FieldUsage\"",
            "default": null,
            "nullable": false,
            "unique": false,
            "comment": "",
            "postgresType": "enum",
            "possibleValues": ["TOP_HALF", "BOTTOM_HALF", "FULL_FIELD"]
          }
        },
        "indexes": {
          "Field_pkey": {
            "name": "Field_pkey",
            "unique": true,
            "columns": ["id"]
          }
        },
        "primaryKey": ["id"],
        "foreignKeys": {},
        "checkConstraints": {},
        "uniqueConstraints": {},
        "xataCompatible": false
      }
    }
  }
}

Other considerations (very hand-wavey)

  1. We can add "checkConstraint" too, which has the PostgreSQL check, e.g. {"checkConstraint": {"length(xata_id) < 256"}}. The frontend can then potentially parse this into a TypeScript validation rule.
  2. Knowing that a type is array, composite, range and their dimensions. Currently, we interpret this in the frontend with some lazy parsing, pgroll should be the source of truth for this information.
  3. Knowing that a type is domain type
  4. Knowing that a type is a custom type
  5. With all these considerations, the output of postgresType can be 'enum' | 'array' | 'range' | 'composite' | 'domain' | 'custom-type'

For the purpose of this ticket, enum is enough.

Related #376

@andrew-farries andrew-farries added the good first issue Good for newcomers label Oct 15, 2024
@ryanslade
Copy link
Collaborator

@divyenduz The enum specific issue has been fixed by #443

@agedemenli
Copy link
Contributor

for the first item in the list, which is:

We can add "checkConstraint" too, which has the PostgreSQL check, e.g. {"checkConstraint": {"length(xata_id) < 256"}}. The frontend can then potentially parse this into a TypeScript validation rule.

Looks like we already have a checkConstraints part in the output, which gives definitions of all check constraints defined on that table. It doesn't make much sense to me to add it again for the columns, as these constraints are defined on table level. Also, a constraint might depend on multiple columns. In that case should we add it into each column definition in the json separately?

I thought maybe this was requested before we had checkConstraints at the table level. Looking into repo history, that's not the case. Looks like checkConstraints was added like a year ago, by @andrew-farries .

One other thing: In case we still want to add constraints at column level, should we remove it from table level?

@andrew-farries
Copy link
Collaborator

I think keeping checkConstraints at the table level is the right thing to do for the reasons you mention.

Looking at the 'other considerations' section, it seems as though we don't currently report the array dimensions as part of the schema:

{
  "name": "01_create_table",
  "operations": [
    {
      "create_table": {
        "name": "items",
        "columns": [
          {
            "name": "id",
            "type": "serial",
            "pk": true
          },
          {
            "name": "names",
            "type": "text[3]",
          }
        ]
      }
    }
  ]
}
resulting schema
+-----------------------------------------------------------------------------------------------------+
| jsonb_pretty                                                                                        |
|-----------------------------------------------------------------------------------------------------|
| {                                                                                                   |
|     "name": "public",                                                                               |
|     "tables": {                                                                                     |
|         "items": {                                                                                  |
|             "oid": "16415",                                                                         |
|             "name": "items",                                                                        |
|             "columns": {                                                                            |
|                 "id": {                                                                             |
|                     "name": "id",                                                                   |
|                     "type": "integer",                                                              |
|                     "unique": true,                                                                 |
|                     "comment": null,                                                                |
|                     "default": "nextval('public.items_id_seq'::regclass)",                          |
|                     "nullable": false,                                                              |
|                     "enumvalues": null                                                              |
|                 },                                                                                  |
|                 "names": {                                                                          |
|                     "name": "names",                                                                |
|                     "type": "text[]",                                                               |
|                     "unique": false,                                                                |
|                     "comment": null,                                                                |
|                     "default": null,                                                                |
|                     "nullable": false,                                                              |
|                     "enumvalues": null                                                              |
|                 }                                                                                   |
|             },                                                                                      |
|             "comment": null,                                                                        |
|             "indexes": {                                                                            |
|                 "items_pkey": {                                                                     |
|                     "name": "items_pkey",                                                           |
|                     "method": "btree",                                                              |
|                     "unique": true,                                                                 |
|                     "columns": [                                                                    |
|                         "id"                                                                        |
|                     ],                                                                              |
|                     "predicate": null,                                                              |
|                     "definition": "CREATE UNIQUE INDEX items_pkey ON public.items USING btree (id)" |
|                 }                                                                                   |
|             },                                                                                      |
|             "primaryKey": [                                                                         |
|                 "id"                                                                                |
|             ],                                                                                      |
|             "foreignKeys": {                                                                        |
|             },                                                                                      |
|             "checkConstraints": {                                                                   |
|             },                                                                                      |
|             "uniqueConstraints": {                                                                  |
|             }                                                                                       |
|         }                                                                                           |
|     }                                                                                               |
| }                                                                                                   |
+-----------------------------------------------------------------------------------------------------+

Postgres doesn't enforce array sizes but I think we should not lose that information, so perhaps we could do that next.

@agedemenli
Copy link
Contributor

I have pushed changes to add check constraints on column level: #584
without removing if from table level.
Let's keep it there in case we need in the future. We can discuss with @divyenduz when he's back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants