Feature: Allow PR status checks to be managed external to safe settings

[jitran]

Background

There have been a number of requests from users who have implemented branch protection rules and rulesets across their organisations, but would like their PR status checks to be defined outside of safe settings.

When these rules are defined at the repository or suborg level, it's not so bad, as the safe settings PR status checks only affect a subset of repositories. Users have a bit more control over their safe settings configuration.

When the rules are defined at the org level, it's a bit more problematic, as the safe settings PR status checks affect all repositories in the org.

Changes

• Added the ability for branch protection rules and rulesets' status checks to be managed outside of safe settings.
• When users specify {{EXTERNALLY_DEFINED}} in the status checks for their branch protection rules or rulesets, safe settings will not revert status checks in GitHub.
• Created an Overrides module to handle externally defined settings.
• Updated usage documentation in the README.
• Added unit tests for the new functionality.

Tests

Ensured all unit tests pass.

Manual testing with rulesets (repo, suborg, and org levels)

The following was tested in sequential order:

1. Started with no rulesets in GitHub for a repository
2. Created a new safe setting for a ruleset that had {{EXTERNALLY_DEFINED}}defined in the status checks
   • Ruleset was deployed with an empty list of status checks
3. Added custom status checks to the new ruleset via GitHub UI
   • The ruleset's custom status checks in GitHub were preserved
4. Updated the ruleset's status checks via the GitHub UI
   • The ruleset's custom status checks in GitHub were preserved
5. Added a new status check to the existing ruleset in safe settings and left {{EXTERNALLY_DEFINED}} in the list
   • The ruleset's custom status checks in GitHub were preserved
6. Removed all status checks from the ruleset via the GitHub UI
   • The ruleset's empty status checks in GitHub was preserved
7. Removed {{EXTERNALLY_DEFINED}} from the existing ruleset's status checks in safe settings
   • The ruleset's status checks in GitHub was reverted to the status checks from safe settings
8. Removed all status checks from the existing ruleset in safe settings
   • The ruleset's status checks in GitHub was reverted to the status checks from safe settings
9. Removed the ruleset from GitHub
   • Safe settings restored the ruleset
10. Added custom status checks to the ruleset via GitHub UI
    • The ruleset's status checks was reverted to the status checks from safe settings
11. Added a completely new ruleset (branch|tag|push) via GitHub UI
    • Safe settings removed all new rulesets

Manual testing with branch protection rules (repo, suborg, and org levels)

The following was tested in sequential order:

1. Deleted all branch protection rules from a repository
2. Created two branch protection rules in safe settings; the first rule had {{EXTERNALLY_DEFINED}} in the status checks, and the second rule had two status checks defined
   • Two branch protection rules were deployed into GitHub, where the first had no status checks, and the second had two status checks defined.
3. Updated the status checks for the first rule via GitHub UI
   • The custom status checks for the first rule in GitHub were preserved
4. Updated the status checks for the second rule via GitHub UI
   • The custom status checks for the second rule in GitHub were reverted back to the safe settings

[Copilot]

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

lib/plugins/branches.js:53

• Redundant check for 'protection.allow_deletions'. It should be simplified to 'protection.allow_deletions = protection.allow_deletions && protection.allow_deletions.enabled'.

protection.allow_deletions = protection.allow_deletions && protection.allow_deletions && protection.allow_deletions.enabled

Tip: Leave feedback on Copilot's review comments with the 👎 and 👍 buttons to help improve review quality. Learn more

[igorcosta]

@jitran , Thanks for the detailed work on this feature! I’ve reviewed the changes, and overall, this is a solid implementation that brings much-needed flexibility to managing PR status checks. That said, I have a few suggestions to improve error handling, documentation clarity, and test coverage:

• In branches.js and rulesets.js, there are scenarios where API failures (e.g., 500-series errors) are logged but not surfaced to the user. Consider adding retry logic for transient errors and ensuring failures are gracefully handled with actionable error messages.

• Include a warning about potential desync when rulesets are modified directly via the GitHub UI.

• In overrides.js, it would be good to add safeguards in removeOverrides to handle malformed configurations or unexpected data structures more robustly.

• Add an example showing how {{EXTERNALLY_DEFINED}} interacts with hierarchical rulesets (org, repo, and sub-org levels).

Let me know your thoughts on these suggestions!

Nice job!