fix(BED-5436): Constraint violations in LinkWellKnownGroups during ingestion #1190
+430
−14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
neumachen
changed the title
neumachen
force-pushed
the
bug/bed-5436
branch
3 times, most recently
from
73c4912 to
4d0130a
Compare
This commit fixes an issue where the LinkWellKnownGroups function was encountering graph DB constraint violations during ingestion. The errors occurred when attempting to create well-known group nodes with SIDs that already existed in the database but with different node types. The root cause was in the getOrCreateWellKnownGroup function, which was filtering nodes by both ObjectID and the Group kind. When a node with the matching ObjectID existed but didn't have the Group kind, the function would try to create a new node with the same ObjectID, causing a constraint violation. The fix modifies the getOrCreateWellKnownGroup function to: 1. Only filter by ObjectID when looking up nodes, not by kind 2. Add the Group kind to existing nodes if it's missing 3. Update the node to ensure it has the correct properties This approach ensures that: - We don't attempt to create duplicate nodes with the same ObjectID - Existing nodes are properly enhanced with the Group kind when needed - We avoid constraint violations by working with existing nodes Additional changes: - Added comprehensive tests to verify the behavior with various node scenarios - Fixed formatting in FixWellKnownNodeTypes This change ensures that SID linking will no longer error when it encounters nodes with different types than expected, addressing the reported constraint violation errors during ingestion. Fixes: [BED-5436](https://specterops.atlassian.net/browse/BED-5436)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes an issue where the LinkWellKnownGroups function encountered graph DB constraint violations during ingestion. The function failed when attempting to create well-known group nodes with SIDs existing in the database but with different node types.
The fix modifies the getOrCreateWellKnownGroup function to:
Motivation and Context
This PR addresses: BED-5436
During ingestion, users encountered errors when the LinkWellKnownGroups function attempted to run. The specific error was a Neo4j constraint violation (Neo.ClientError.Schema.ConstraintValidationFailed) because nodes with the same SID but different types already existed in the database. This could happen due to previous incomplete or incorrect ingestions.
The solution ensures that SID linking will work correctly even when nodes with the same SID but different types already exist, preventing constraint violations and allowing ingestion to complete successfully.
How Has This Been Tested?
Added comprehensive tests that verify the behavior with various node scenarios:
Tests are run against Neo4j and PostgreSQL databases to ensure compatibility across supported database types.
Types of changes
Checklist: