JSON Column Data Altered During Switchover

[mastercactapus]

Describe the Bug:
The data contained within json-type columns seems to get altered during a switchover operation. Specifically, the order of keys in the JSON policy column of gql_api_keys changes after performing a switchover.

Steps to Reproduce:

1. Run GoAlert commit a25f8082e975812ff5d4b88416768cd931aaa34c or earlier (using make start-swo)
2. Create a GraphQL API key and verify it functions
3. Perform a switchover (Admin > Switchover)
4. Observe that the API key no longer works

Expected Behaviour:
For json-type columns, the data should be preserved exactly during switchover operations. This includes preventing change to the order of keys within the JSON documents stored within these columns.

Observed Behavior:
After performing a switchover, the order of keys in the JSON data changes. This suggests that the data might be parsed into a map and then re-encoded, rather than being copied as a json.RawMessage.

Application Version:
This issue affects all known versions up to and including the commit a25f8082e975812ff5d4b88416768cd931aaa34c.

Additional Context:
This issue was identified due to GraphQL API keys being rejected following a switchover, as the data alteration resulted in a hash mismatch.

This issue specifically affects the backend.

[Root-acess]

The bug described here appears to be a result of how JSON data is handled during switchover operations, particularly due to an unintended re-encoding of JSON that alters key order. This causes the API keys to become invalid as any change to the JSON structure, including the order of keys, can lead to a hash mismatch and rejection by the application.

Analysis of the Issue

• Cause: When GoAlert performs a switchover, JSON data in certain columns, such as the policy column within gql_api_keys, is likely being read into a map or struct in Go, which doesn’t preserve the original key order. When this data is subsequently re-encoded into JSON, the key order changes, affecting the stored hash and making API keys invalid.

• Implication: JSON key ordering affects hash validation, meaning any alteration can render the API key unusable. JSON is technically an unordered format, but for systems that rely on exact hashing of serialized JSON, maintaining key order is critical.

Steps to Reproduce the Issue

1. Set up Environment: Use GoAlert commit a25f8082e975812ff5d4b88416768cd931aaa34c or earlier.
2. Create and Validate API Key: Generate a GraphQL API key and confirm it is functional.
3. Perform Switchover: Navigate to the admin panel and initiate a switchover (Admin > Switchover).
4. Check Key Functionality: After switchover, try using the API key again, and observe that it fails due to a hash mismatch.

Solution

To ensure the JSON data remains unchanged during switchover operations, the backend should avoid parsing JSON data from the database into a map or struct if order preservation is critical. Instead, it should be copied directly as a json.RawMessage, which preserves the original serialized data without alteration.

Implementation Steps

1. Store JSON as json.RawMessage:

   • Update any code that reads from the policy column (or similar JSON-type columns) during switchover operations to treat JSON data as json.RawMessage.
   • This ensures the exact serialized form is copied, preserving key order and preventing hash mismatches.

2. Update Hash Validation:

   • Modify the backend code that performs hash comparisons on JSON fields to accommodate the unmodified json.RawMessage values.

3. Testing:

   • Verify that API keys continue to function after performing a switchover.
   • Conduct tests on other JSON columns to ensure they are unaffected by similar operations.

Expected Outcome

After implementing these changes, switchover operations should no longer alter the JSON key order in gql_api_keys or similar columns, preserving hash integrity and ensuring API keys remain functional post-switchover.

Additional

1. Documentation: Update GoAlert’s internal documentation to clarify the handling of JSON columns during switchover and other operations, particularly regarding key ordering requirements for hashing.
2. Future Safeguards: Consider introducing automated tests to detect unintended JSON changes in sensitive columns during switchover operations.