Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We currently use JSON to store all synced data, with views to present it as a normal table.
Part of this is using triggers to persist updates (insert/update/delete), using the SQLite
json_object
function to convert the data back to JSON.The issue is that SQLite limits function calls to 100 arguments by default, and this can be increased to a maximum of 127. Since
json_object
requires 2 arguments for each value (key + value), this means we can have a maximum of 50 or 63 columns, depending on the value of SQLITE_MAX_FUNCTION_ARG.The change here works around the issue by building up the json object in chunks of 50 columns, then merge together using a new
powersync_json_merge
function. This takes the form ofpowersync_json_merge(json_object('a', 1, 'b', 2), json_object('c', 3, 'd', 4))
. This is slightly less efficient than a single json_object call, so we only do this when there are more than 50 columns in a table.With SQLITE_MAX_FUNCTION_ARG = 100, this would give us a maximum of
100 * 50 = 5000
columns. However, SQLITE_MAX_COLUMN with a default of 2000 is still a limiting factor, so we have a practical limit of 1999 columns (excluding id).