Ability to create suborgs using custom properties. (#624)

* Add support for suborgs using custom properties

* update README and app.yml

* upgrade node 18

* upgrade node 18

* reset probot version for Docker build to work

* Update README.md

* Update README.md

README.md

@@ -5,7 +5,7 @@
 `Safe-settings`– an app to manage policy-as-code and apply repository settings to repositories across an organization.
 
 1. In `safe-settings` all the settings are stored centrally in an `admin` repo within the organization. This is important. Unlike [Settings Probot](https://github.com/probot/settings), the settings files cannot be in individual repositories.  
-    > **Note**
+    > [!Note]
     > It is possible to override this behavior and specify a custom repo instead of the `admin` repo.<br>
     > This could be done by setting an `env` variable called `ADMIN_REPO`.
 
@@ -16,19 +16,23 @@
 
 3. For The `repo`-targeted settings there can be at 3 levels at which the settings could be managed:
    1. Org-level settings are defined in `.github/settings.yml`  
-       > **Note**
+       > [!Note]
        > It is possible to override this behavior and specify a different filename for the `settings` yml repo.<br>
        > This could be done by setting an `env` variable called `SETTINGS_FILE_PATH`.<br>
        > Similarly, the `.github` directory can be overridden with an `env` variable called `CONFIG_PATH`.
 
-   2. `Suborg` level settings. A `suborg` is an arbitrary collection of repos belonging to projects, business units, or teams. The `suborg` settings reside in a yaml file for each `suborg` in the `.github/suborgs` folder.
+   2. `Suborg` level settings. A `suborg` is an arbitrary collection of repos belonging to projects, business units, or teams. The `suborg` settings reside in a yaml file for each `suborg` in the `.github/suborgs` folder. 
+
+   > [!Note]
+   > In `safe-settings`, sub orgs could be groups of repos based on `repo names`, or `teams` which the repos have collaborators from, or `custom property values` set for the repos
+
    3. `Repo` level settings. They reside in a repo specific yaml in `.github/repos` folder
 4. It is recommended to break the settings into org-level, suborg-level, and repo-level units. This will allow different teams to define and manage policies for their specific projects or business units. With `CODEOWNERS`, this will allow different people to be responsible for approving changes in different projects.
 
-> **Note**
+> [!Note]
 > `Suborg` and `Repo` level settings directory structure cannot be customized.
 
-> **Note**
+> [!Note]
 > The settings file must have a `.yml` extension only. `.yaml` extension is ignored, for now.
 
 ## How it works
@@ -59,7 +63,7 @@ To apply `safe-settings` __only__ to a specific list of repos, add them to the `
 
 To ignore `safe-settings` for a specific list of repos, add them to the `restrictedRepos` section as `exclude` array.
 
-> **Note**
+> [!Note]
 > The `include` and `exclude` attributes support as well regular expressions.
 > By default they look for regex, Example include: ['SQL'] will look apply to repos with SQL and SQL_ and SQL- etc if you want only SQL repo then use include:['^SQL$']
 
@@ -237,7 +241,7 @@ For e.g. If we have `override` validators that will fail if `org-level` branch p
 <img width="467" alt="image" src="https://github.com/github/safe-settings/assets/57544838/cc5d59fb-3d7c-477b-99e9-94bcafd07c0b">
 </p>
 
-> **NOTE**
+> [!NOTE]
 > If you don't want the PR message to have these details, it can be turned off by `env` setting `CREATE_PR_COMMENT`=`false`
 
 Here is a screenshot of what the users will see in the `checkrun` page:

app.yml

@@ -13,44 +13,25 @@
 # The list of events the GitHub App subscribes to.
 # Uncomment the event names below to enable them.
 default_events:
+  - custom_property_values
+  - repository_ruleset
   - check_run
   - check_suite
   - branch_protection_rule
-  # - commit_comment
-  # - create
-  # - delete
-  # - deployment
-  # - deployment_status
-  # - fork
-  # - gollum
-  # - issue_comment
-  # - issues
-  # - label
-  # - milestone
   - member
-  # - membership
-  # - org_block
-  # - organization
-  # - page_build
-  # - project
-  # - project_card
-  # - project_column
-  # - public
   - pull_request
   - push
-  # - release
   - repository
-  # - repository_import
-  # - status
   - team
-  # - team_add
-  # - watch
+
 
 # The set of permissions needed by the GitHub App. The format of the object uses
 # the permission name for the key (for example, issues) and the access type for
 # the value (for example, write).
 # Valid values are `read`, `write`, and `none`
 default_permissions:
+    organization_custom_properties: admin
+
     # Repository creation, deletion, settings, teams, and collaborators.
     # https://developer.github.com/v3/apps/permissions/#permission-on-administration
     administration: write

docs/sample-settings/suborg.yml

@@ -5,9 +5,14 @@ suborgrepos:
   - test*
 # You can use Glob patterns
 
+# List of repos that belong to the suborg
 suborgteams:
   - core
 
+# List of repos that belong to the suborg based on custom properties
+suborgproperties:     
+- EDP: true
+
 #repository: 
   # This is the settings that need to be applied to all repositories in the org 
   # See https://developer.github.com/v3/repos/#edit for all available settings for a repository  

index.js

@@ -189,6 +189,26 @@ module.exports = (robot, { getRouter }, Settings = require('./lib/settings')) =>
     robot.log.debug(JSON.stringify(res, null))
   }
 
+  async function info() {
+    const github = await robot.auth()
+    const installations = await github.paginate(
+      github.apps.listInstallations.endpoint.merge({ per_page: 100 })
+    )
+    robot.log.debug(`installations: ${JSON.stringify(installations)}`)
+    if (installations.length > 0) {
+      const installation = installations[0]
+      robot.log.debug(`Installation ID: ${installation.id}`)
+      robot.log.debug('Fetching the App Details')
+      const github = await robot.auth(installation.id)
+      const app = await github.apps.getAuthenticated()
+      robot.log.debug(`Validated the app is configured properly = \n${JSON.stringify(app.data, null, 2)}`)
+      robot.log.debug(`Registered App name = ${app.data.slug}\n`)
+      robot.log.debug(`Permissions = ${JSON.stringify(app.data.permissions)}\n`)
+      robot.log.debug(`Events = ${app.data.events}\n`)
+    }
+  }
+
+
   async function syncInstallation () {
     robot.log.trace('Fetching installations')
     const github = await robot.auth()
@@ -283,6 +303,18 @@ module.exports = (robot, { getRouter }, Settings = require('./lib/settings')) =>
     return syncSettings(false, context)
   })
 
+  robot.on('custom_property_values', async context => {
+    const { payload } = context
+    const { sender } = payload
+    robot.log.debug('Custom Property Value Updated for a repo by ', JSON.stringify(sender))
+    if (sender.type === 'Bot') {
+      robot.log.debug('Custom Property Value edited by Bot')
+      return
+    }
+    robot.log.debug('Custom Property Value edited by a Human')
+    return syncSettings(false, context)
+  })
+
   robot.on('repository_ruleset', async context => {
     const { payload } = context
     const { sender } = payload
@@ -525,6 +557,9 @@ module.exports = (robot, { getRouter }, Settings = require('./lib/settings')) =>
       syncInstallation()
     })
   }
+
+  //Uncomment below to get info about the app configuration
+  //info()
 
   return {
     syncInstallation

lib/plugins/repository.js

@@ -89,6 +89,14 @@ module.exports = class Repository extends ErrorStash {
           resArray.push(new NopCommand('Repository', this.repo, null, topicResults))
         }
         const promises = []
+        if (topicChanges.hasChanges) {
+          promises.push(this.updatetopics(resp.data, resArray))
+        } else {
+          this.log.debug(`There are no changes for repo ${JSON.stringify(this.repo)}.`)
+          if (this.nop) {
+            resArray.push(new NopCommand('Repository', this.repo, null, `There are no changes for repo ${JSON.stringify(this.repo)}.`))
+          }
+        }
         if (changes.hasChanges) {
           this.log.debug('There are repo changes')
           let updateDefaultBranchPromise = Promise.resolve()
@@ -108,14 +116,6 @@ module.exports = class Repository extends ErrorStash {
         } else {
           promises.push(this.updateSecurity(resp.data, resArray))
         }
-        if (topicChanges.hasChanges) {
-          promises.push(this.updatetopics(resp.data, resArray))
-        } else {
-          this.log.debug(`There are no changes for repo ${JSON.stringify(this.repo)}.`)
-          if (this.nop) {
-            resArray.push(new NopCommand('Repository', this.repo, null, `There are no changes for repo ${JSON.stringify(this.repo)}.`))
-          }
-        }
         if (this.nop) {
           return Promise.resolve(resArray)
         } else {
@@ -125,7 +125,7 @@ module.exports = class Repository extends ErrorStash {
         if (e.status === 404) {
           if (this.force_create) {
             if (this.template) {
-              this.log(`Creating repo using template ${this.template}`)
+              this.log.debug(`Creating repo using template ${this.template}`)
               const options = { template_owner: this.repo.owner, template_repo: this.template, owner: this.repo.owner, name: this.repo.repo, private: (this.settings.private ? this.settings.private : true), description: this.settings.description ? this.settings.description : '' }
 
               if (this.nop) {
@@ -138,7 +138,7 @@ module.exports = class Repository extends ErrorStash {
               // https://docs.github.com/en/rest/repos/repos#create-an-organization-repository uses org instead of owner like
               // the API to create a repo with a template
               this.settings.org = this.settings.owner
-              this.log('Creating repo with settings ', this.settings)
+              this.log.debug('Creating repo with settings ', this.settings)
               if (this.nop) {
                 this.log.debug(`Creating Repo ${JSON.stringify(this.github.repos.createInOrg.endpoint(this.settings))}  `)
                 return Promise.resolve([
@@ -220,30 +220,30 @@ module.exports = class Repository extends ErrorStash {
     }
 
     if (this.topics) {
-      if (repoData.data?.topics.length !== this.topics.length ||
-        !repoData.data?.topics.every(t => this.topics.includes(t))) {
-        this.log(`Updating repo with topics ${this.topics.join(',')}`)
+      // if (repoData.data?.topics.length !== this.topics.length ||
+      //   !repoData.data?.topics.every(t => this.topics.includes(t))) {
+        this.log.debug(`Updating repo with topics ${this.topics.join(',')}`)
         if (this.nop) {
           resArray.push((new NopCommand(this.constructor.name, this.repo, this.github.repos.replaceAllTopics.endpoint(parms), 'Update Topics')))
           return Promise.resolve(resArray)
         }
         return this.github.repos.replaceAllTopics(parms)
-      } else {
-        this.log(`no need to update topics for ${repoData.data.name}`)
-        if (this.nop) {
-          //resArray.push((new NopCommand(this.constructor.name, this.repo, null, `no need to update topics for ${repoData.data.name}`)))
-          return Promise.resolve([])
-        }
-      }
+      // } else {
+      //   this.log.debug(`no need to update topics for ${repoData.data.name}`)
+      //   if (this.nop) {
+      //     //resArray.push((new NopCommand(this.constructor.name, this.repo, null, `no need to update topics for ${repoData.data.name}`)))
+      //     return Promise.resolve([])
+      //   }
+      // }
     }
   }
 
   // Added support for Code Security and Analysis
   updateSecurity (repoData, resArray) {
     if (this.security?.enableVulnerabilityAlerts === true || this.security?.enableVulnerabilityAlerts === false) {
-      this.log(`Found repo with security settings ${JSON.stringify(this.security)}`)
+      this.log.debug(`Found repo with security settings ${JSON.stringify(this.security)}`)
       if (this.security.enableVulnerabilityAlerts === true) {
-        this.log(`Enabling Dependabot alerts for owner: ${repoData.owner.login} and repo ${repoData.name}`)
+        this.log.debug(`Enabling Dependabot alerts for owner: ${repoData.owner.login} and repo ${repoData.name}`)
         if (this.nop) {
           resArray.push((new NopCommand(this.constructor.name, this.repo, this.github.repos.enableVulnerabilityAlerts.endpoint({
             owner: repoData.owner.login,
@@ -256,7 +256,7 @@ module.exports = class Repository extends ErrorStash {
           repo: repoData.name
         })
       } else {
-        this.log(`Disabling Dependabot alerts for for owner: ${repoData.owner.login} and repo ${repoData.name}`)
+        this.log.debug(`Disabling Dependabot alerts for for owner: ${repoData.owner.login} and repo ${repoData.name}`)
         if (this.nop) {
           resArray.push((new NopCommand(this.constructor.name, this.github.repos.disableVulnerabilityAlerts.endpoint({
             owner: repoData.owner.login,
@@ -270,7 +270,7 @@ module.exports = class Repository extends ErrorStash {
         })
       }
     } else {
-      this.log(`no need to update security for ${repoData.name}`)
+      this.log.debug(`no need to update security for ${repoData.name}`)
       if (this.nop) {
         //resArray.push((new NopCommand(this.constructor.name, this.repo, null, `no need to update security for ${repoData.name}`)))
         return Promise.resolve([])
@@ -281,7 +281,7 @@ module.exports = class Repository extends ErrorStash {
   updateAutomatedSecurityFixes (repoData, resArray) {
     if (this.security?.enableAutomatedSecurityFixes === true || this.security?.enableAutomatedSecurityFixes === false) {
       if (this.security.enableAutomatedSecurityFixes === true) {
-        this.log(`Enabling Dependabot security updates for owner: ${repoData.owner.login} and repo ${repoData.name}`)
+        this.log.debug(`Enabling Dependabot security updates for owner: ${repoData.owner.login} and repo ${repoData.name}`)
         if (this.nop) {
           resArray.push((new NopCommand(this.constructor.name, this.repo, this.github.repos.enableAutomatedSecurityFixes.endpoint({
             owner: repoData.owner.login,
@@ -294,7 +294,7 @@ module.exports = class Repository extends ErrorStash {
           repo: repoData.name
         })
       } else {
-        this.log(`Disabling Dependabot security updates for owner: ${repoData.owner.login} and repo ${repoData.name}`)
+        this.log.debug(`Disabling Dependabot security updates for owner: ${repoData.owner.login} and repo ${repoData.name}`)
         if (this.nop) {
           resArray.push((new NopCommand(this.constructor.name, this.github.repos.disableAutomatedSecurityFixes.endpoint({
             owner: repoData.owner.login,

lib/settings.js

@@ -711,6 +711,18 @@ ${this.results.reduce((x, y) => {
             })
           })
         }
+        if (data.suborgproperties) {
+          const promises = data.suborgproperties.map((customProperty) => {
+            return this.getReposForCustomProperty(customProperty)
+          })
+          await Promise.all(promises).then(res => {
+            res.forEach(r => {
+              r.forEach(e => {
+                subOrgConfigs[e.repository_name] = data
+              })
+            })
+          })
+        }
       }
       return subOrgConfigs
     } catch (e) {
@@ -783,6 +795,15 @@ ${this.results.reduce((x, y) => {
     return this.github.paginate(options)
   }
 
+  async getReposForCustomProperty (customPropertyTuple) {
+    const name=Object.keys(customPropertyTuple)[0]
+    let q = `props.${name}:${customPropertyTuple[name]}`
+    q = encodeURIComponent(q)
+    const options = this.github.request.endpoint((`/orgs/${this.repo.owner}/properties/values?repository_query=${q}`))
+    return this.github.paginate(options)
+  }
+
+
   isObject (item) {
     return (item && typeof item === 'object' && !Array.isArray(item))
   }