Skip to content

Commit

Permalink
[CI] Split variables to reduce needed rights. (#21371)
Browse files Browse the repository at this point in the history
Split the variables and document their usage. Because we are going to
have to add more pipelines we want to limit the amount of secrets a new
pipeline requests and uses.

The split is as follows:

- variables/common.yml: needed by all.
- variables/api-scan.yml: needed by the api scan pipeline.
- variables/signing.yml: needed by pipeline that need to perform
signing.

---------

Co-authored-by: Rolf Bjarne Kvinge <[email protected]>
  • Loading branch information
mandel-macaque and rolfbjarne authored Oct 4, 2024
1 parent b7a317d commit 8bb8f6f
Show file tree
Hide file tree
Showing 11 changed files with 85 additions and 71 deletions.
2 changes: 1 addition & 1 deletion tools/devops/automation/build-cronjob.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ parameters:
default: false

variables:
- template: templates/variables.yml
- template: templates/variables/common.yml
- name: MaciosUploadPrefix
value: ''

Expand Down
4 changes: 1 addition & 3 deletions tools/devops/automation/build-lego.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,7 @@ parameters:
default: true

variables:
- template: templates/variables.yml
- name: MaciosUploadPrefix
value: ''
- template: templates/variables/common.yml

resources:
repositories:
Expand Down
6 changes: 3 additions & 3 deletions tools/devops/automation/build-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -169,11 +169,11 @@ variables:
- ${{ if contains(variables['Build.DefinitionName'], 'private') }}:
- template: templates/vsts-variables.yml
- template: templates/common/vs-release-vars.yml@sdk-insertions
- template: templates/variables.yml
- template: templates/variables/common.yml
- template: templates/variables/api-scan.yml
- template: templates/variables/signing.yml
- name: MicrobuildConnector
value: 'MicroBuild Signing Task (DevDiv)'
- name: MaciosUploadPrefix
value: ''
- name: DisablePipelineConfigDetector
value: true

Expand Down
7 changes: 2 additions & 5 deletions tools/devops/automation/build-pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -157,12 +157,9 @@ resources:
name: 1ESPipelineTemplates/MicroBuildTemplate

variables:
- template: templates/variables.yml
- template: templates/variables/common.yml
- template: templates/variables/api-scan.yml
- template: templates/common/vs-release-vars.yml@sdk-insertions
- name: MicrobuildConnector
value: ''
- name: MaciosUploadPrefix
value: ''
- name: Packaging.EnableSBOMSigning
value: false

Expand Down
16 changes: 5 additions & 11 deletions tools/devops/automation/templates/pipelines/api-diff-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,18 +57,12 @@ resources:
endpoint: xamarin

variables:
- ${{ if eq(parameters.isPR, false) }}:
- ${{ if contains(variables['Build.DefinitionName'], 'private') }}:
- template: ../vsts-variables.yml
- template: ../variables.yml
- name: MaciosUploadPrefix
value: ''
- name: DisablePipelineConfigDetector
- template: ../variables/common.yml
- name: DisablePipelineConfigDetector
${{ if eq(parameters.isPR, false) }}:
value: true
- ${{ else }}:
- template: ../variables.yml
- name: MaciosUploadPrefix
value: ''
${{ else }}:
value: false

stages:
- template: ../api-diff-stage.yml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -202,11 +202,7 @@ resources:
endpoint: xamarin

variables:
- ${{ if contains(variables['Build.DefinitionName'], 'private') }}:
- template: ../vsts-variables.yml
- template: ../variables.yml
- name: MaciosUploadPrefix
value: ''
- template: ../variables/common.yml
- name: DisablePipelineConfigDetector
value: true

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,7 @@ resources:
endpoint: xamarin

variables:
- ${{ if contains(variables['Build.DefinitionName'], 'private') }}:
- template: ../vsts-variables.yml
- template: ../variables.yml
- name: MaciosUploadPrefix
value: ''
- template: ../variables/common.yml
- name: DisablePipelineConfigDetector
value: true

Expand Down
38 changes: 0 additions & 38 deletions tools/devops/automation/templates/variables.yml

This file was deleted.

3 changes: 3 additions & 0 deletions tools/devops/automation/templates/variables/api-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
variables:
- name: TeamName
value: 'xamarin-macios'
61 changes: 61 additions & 0 deletions tools/devops/automation/templates/variables/common.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
variables:
# provisionator-uri setting
- group: XamarinCompatLab

# allow to override the provisionator channel via a paramter
- name: PROVISIONATOR_CHANNEL
value: ${{ parameters.provisionatorChannel }}

- group: xamops-azdev-secrets

# needed to install the dev certificates in the build machine
- group: Xamarin Notarization
- group: Xamarin Signing

# Override the GitHub.Token setting defined in the Xamarin Release group
# Use a token dedicated to critical production workflows and help avoid GitHub throttling
# this variable comes from the Xamarin-Secrets group, so we have do have both together
- group: Xamarin-Secrets
- name: GitHub.Token
value: $(github--pat--vs-mobiletools-engineering-service2)

# Overrude the azdo build access, as with the GitHub token, this variables depend on the
# Xamarin-Secrets group.
- name: AzDoBuildAccess.Token
value: $(pat--xamarinc--build-access)

- name: AzDoBuildAccess.Token
value: $(pat--xamarinc--build-access)

# keying to use for the sining
- name: SigningKeychain
value: "builder.keychain"

# Needed to ensure that devices uses the usb cable to communicate with the devices to run the tests.
- name: USE_TCP_TUNNEL
value: true

# pool information
- name: PRBuildPool
value: 'VSEng-Xamarin-RedmondMacBuildPool-iOS-Untrusted'
- name: PRBuildPoolUrl
value: 'https://devdiv.visualstudio.com/_settings/agentpools?poolId=366&view=agents'
- name: CIBuildPool
value: 'VSEng-Xamarin-RedmondMacBuildPool-iOS-Trusted'
- name: CIBuildPoolUrl
value: 'https://devdiv.visualstudio.com/_settings/agentpools?poolId=367&view=agents'

# override the default build revision
- name: BUILD_REVISION
value: azure-devops-$(Build.SourceVersion)

- name: MaciosUploadPrefix
value: ''

# set the pipeline to debug mode or not
- name: system.debug
value: false

# point to the vsdrops that we will be using for the diff uploads.
- name: VSDropsPrefix
value: 'https://vsdrop.corp.microsoft.com/file/v1/xamarin-macios/device-tests'
7 changes: 7 additions & 0 deletions tools/devops/automation/templates/variables/signing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# signing related variables
variables:
- group: Xamarin-Secrets
- group: Xamarin Signing
- group: Xamarin Release
- group: Xamarin Notarization
- group: VSEng DTL secrets

9 comments on commit 8bb8f6f

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 Tests failed catastrophically on VSTS: simulator tests - monotouch - iOS 🔥

Not enough free space in the host.

Pipeline on Agent
[CI] Split variables to reduce needed rights. (#21371)

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📚 [CI Build] Artifacts 📚

Artifacts were not provided.

Pipeline on Agent
Hash: [CI build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

All tests on macOS X64 - Mac Sonoma (14) passed.

Pipeline on Agent
Hash: 8bb8f6f82ce28ec01c6542f5ad93c3a77b203532 [CI build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ API diff for current PR / commit

NET (empty diffs)
  • iOS: (empty diff detected)
  • tvOS: (empty diff detected)
  • MacCatalyst: (empty diff detected)
  • macOS: (empty diff detected)

✅ API diff vs stable

.NET (No breaking changes)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: 8bb8f6f82ce28ec01c6542f5ad93c3a77b203532 [CI build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

All tests on macOS M1 - Mac Monterey (12) passed.

Pipeline on Agent
Hash: 8bb8f6f82ce28ec01c6542f5ad93c3a77b203532 [CI build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

All tests on macOS M1 - Mac Ventura (13) passed.

Pipeline on Agent
Hash: 8bb8f6f82ce28ec01c6542f5ad93c3a77b203532 [CI build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ [CI Build] Windows Integration Tests failed ❌

❌ Failed ❌

Pipeline on Agent
Hash: 8bb8f6f82ce28ec01c6542f5ad93c3a77b203532 [CI build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 [CI Build] Test results 🚀

Test results

✅ All tests passed on VSTS: test results.

🎉 All 99 tests passed 🎉

Tests counts

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 1 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 4 tests passed. Html Report (VSDrops) Download
✅ linker: All 40 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 7 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 8 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 9 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 7 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: 8bb8f6f82ce28ec01c6542f5ad93c3a77b203532 [CI build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 [CI Build] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

3 tests crashed, 0 tests failed, 82 tests passed.

Failures

❌ dotnettests tests (MacCatalyst)

🔥 Failed catastrophically on VSTS: test results - dotnettests_maccatalyst (no summary found).

Html Report (VSDrops) Download

❌ monotouch tests (iOS)

🔥 Failed catastrophically on VSTS: test results - monotouch_ios (no summary found).

Html Report (VSDrops) Download

❌ monotouch tests (macOS)

🔥 Failed catastrophically on VSTS: test results - monotouch_macos (no summary found).

Html Report (VSDrops) Download

Successes

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 1 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 4 tests passed. Html Report (VSDrops) Download
✅ linker: All 40 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 8 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 7 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: 8bb8f6f82ce28ec01c6542f5ad93c3a77b203532 [CI build]

Please sign in to comment.