Set exact id value for List item idField on create mutations

[MurzNN]

Very often developers need to set exact value for id field of creating List item, but Keystone don't allow this now! So please add ability to set id value on List item create mutations in GraphQL.

Most of other frameworks and CMS allow pass value for entity id on creating stage, and have no architectural or security problems with this feature. So I, and, as I think, many other developers are very surprised, when figuring out that this valuable feature is missing in Keystone!

Examples of situations, when it is necessary:

1. Syncing List items with some remote database, for example - Localities (Cities) from OpenStreetMap - each remote item already have unique id in BigInt format, that can be reused in Keystone database.

   The workaround is storing OSM id in separate field like "osm_id", but it is pain to use two ids of one item, especially on frontend side of the app, which uses Keystone API and OSM API, and developer must always remember, that when talking with Keystone he must use value from id field value, with OSM - use osm_id field value. Also on every relation field in Keystone he will receive useless internal id of OSM object, so must make additional query (or join) to receive real osm_id value to reuse in fronted. And so on.

2. Use list as the key-value storage, where key is unique string, value is JSON object. Now all items will have the totally useless id value, additionally to key.

3. Working with list items, that are stored in different databases. For example, two separate Keystone instances: first - for e-shop website, second - separate backend for manage orders. And we need to keep order id in sync in both Keystone instances.

4. Having several lists with identical ids of items. For example, very popular practice is to keep the user list compact and simple, and create separate lists like userAdminData, userStudentData, userProfessorData with many fields, to store additional info about user. And very convenient, when those items have same id as user item, instead of having separate field as relation to user List and separate unique id value.

5. Simple list of Countries with id-name fields, instead of have simple compact two-field table with records like "RU - Russia", "AU - Australia", Keystone forces us to use third useless id field as autoIncrement or uuid.

...and many other situations.

So, please, consider the ways to allow setting desired id value on List item create stage in Keystone!

If implementing this feature can give some security risks or other problems with deep create queries, you always can mark this feature as "Experimental" with limited support and warnings. For most of needs, it will be enough to allow set id value on simple create mutations, that's all!

[MurzNN]

Here is my PR, that tries to implement this feature #4826

[MurzNN]

I reworked this feature in new PR #6048

[MurzNN]

Another common pattern is to store user metadata in different Lists, but with same id as user.

For example, one user have profile as "student" (20 fields), second as "teacher" (30 fields), third as "admin" (15 fields):

Instead of extending List "user" with 65 fields, much better to create separate lists like userProfileStudent, userProfileTeacher, userProfileAdmin.

And very convenient when id of such profile items are exactly same as id of user item, so we can load profile directly from needed List, without more heavy query using relation to user.

[raveling]

Thanks @MurzNN we are in continuing discussions about your proposals here, and in the related PRs.

[molomby]

Hey @MurzNN we've discussed this internally several times in the last few years. The decision to use arbitrary and opaque values as primary keys on all lists is quite intentional. We recognise this precludes some use cases and database structures but Keystone simply can't attempt to solve all use cases for all apps. In our experience, using natural or meaningful keys eventually creates re-keying problems in many systems and should generally be avoided. The use of opaque keys also aligns with the GraphQL spec.

This doesn't prevent developers from storing as many unique, natural keys as they'd like, it just means you can't use them as the primary key. As such, the workaround you've suggested (storing the OSM id in separate field) is the solution we'd recommend. Note that the recent GraphQL schema changes let you perform update and delete operations using any of a lists unique fields (not just the id). As you point out though, relationships must still be managed using the Keystone-generated id values.

Keystone always tries to give developers escape hatches though. If you really want to specify your keys for a specific list there are several other approaches that may work:

• If you want to create a static table of, like in your Countries use case, create a mutation that imports the data in plain SQL
• If you a list is being used by a custom frontend, you could create a custom mutation to handle create operations, that internally used the Prisma client directly. This will let you insert arbitrary id for new items.
• If you want to maintain the ability to use the Admin UI while also using custom IDs on create, a solution may be possible using the afterChange hook. Ie. let Keystone inset the record as usual then use the Prisma client the replace the ID before returning (though this won't allow nested creates, etc. without additional work).
• Likewise, if sync'ing large amounts of existing data from another system (eg. OpenStreetMap), perhaps build your sync operation against Prisma (or the DB directly using DTL or similar), rather than sync'ing via the Keystone GraphQL schema

I'm sure there are other ways too but solving for these use cases (beyond providing the escape hatches and hooks) is outside the scope of the Keystone framework.

[MurzNN]

We still use the custom id values in our several projects, so I've updated my fork to the latest Keystone release (2022-11-21), hope it will be useful for anyone else with the same problem.
The updated fork is here: https://github.com/MurzNN/keystone/tree/idfield-overrides-3

[MurzNN]

In our experience, using natural or meaningful keys eventually creates re-keying problems in many systems and should generally be avoided. The use of opaque keys also aligns with the GraphQL spec.

With uuid and cuid id field types - I agree that pre-filling them is not a good idea (but a good workaround in some cases), but we still have numeric integer keys (autoincrement), why they can't be used to have custom values on create? :(