-
Notifications
You must be signed in to change notification settings - Fork 359
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #4281 from VladimirSlavik/master-templates-ci
CI templates
- Loading branch information
Showing
32 changed files
with
1,456 additions
and
61 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,13 @@ | ||
| branch_type: "test" | ||
| # Settings for infrastructure autogeneration | ||
| # | ||
| # See also: | ||
| # scripts/jinja-render | ||
| # make reload-infra | ||
| # | ||
| # Follow "type hints" below or be sad. The choice is yours. | ||
|
|
||
| distro_name: "fedora" # "fedora" or "rhel" | ||
| distro_release: "rawhide" # "rawhide" or a number without quotation marks | ||
|
|
||
| # The following only applies for rawhide. | ||
| branched_fedora_version: 37 # number without quotation marks, or nothing if CI should not run for branched Fedora |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| Templated workflows | ||
| ------------------- | ||
|
|
||
| Almost all workflows are now built from jinja2 templates. Make sure you are editing the `.j2` | ||
| template, not the rendered `.yml` file. To rebuild the workflow files from templates, run | ||
| `make -f Makefile.am reload-infra` at any time. The rebuild depends only on the | ||
| `.branch-variables.yml` file in the repo root. | ||
|
|
||
| Most of the workflows are triggered by cron or comment events, so they belong only on the default | ||
| branch which is `master`. These workflows are removed by templates on other branches. If the first | ||
| line is `{% if distro_release == "rawhide" %}` then the workflow is of such kind. | ||
|
|
||
| When editing a template, the following roughly describes what to expect: | ||
|
|
||
| - Any values available for the templates come from `.branch-variables.yml` in the repo root. | ||
|
|
||
| - Inline variables `{$ ... $}` are replaced by values. | ||
|
|
||
| - Blocks `{% ... %}` let you use conditions to select which block will be present in the output. | ||
| If you don't put anything else on a line except for the block itself, the line will completely | ||
| disappear from the output. Prefer that and avoid using blocks inline. | ||
|
|
||
| - Whitespace handling is a complicated affair. YAML wants it precise, while Jinja is messy. | ||
| If you stick to the two methods above, everything stays mostly deterministic. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| {% if distro_release == "rawhide" %} | ||
| # Build a boot.iso from a PR triggered by a "/boot-iso" comment or manually. | ||
| name: Build boot.iso | ||
| on: | ||
| issue_comment: | ||
| types: [created] | ||
| # be able to start this action manually from a actions tab when needed | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| statuses: write | ||
|
|
||
| jobs: | ||
| pr-info: | ||
| if: github.event_name == 'workflow_dispatch' || startsWith(github.event.comment.body, '/boot-iso') | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Query comment author repository permissions | ||
| if: github.event_name != 'workflow_dispatch' | ||
| uses: octokit/[email protected] | ||
| id: user_permission | ||
| with: | ||
| route: GET /repos/${{ github.repository }}/collaborators/${{ github.event.sender.login }}/permission | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| # restrict this workflow to users with admin or write permission for the repository | ||
| # see https://docs.github.com/en/free-pro-team@latest/rest/reference/repos#get-repository-permissions-for-a-user | ||
| # store output if user is allowed in allowed_user job output so it has to be checked in downstream job | ||
| - name: Check if user does have correct permissions | ||
| if: github.event_name != 'workflow_dispatch' && contains('admin write', fromJson(steps.user_permission.outputs.data).permission) | ||
| id: check_user_perm | ||
| run: | | ||
| echo "User '${{ github.event.sender.login }}' has permission '${{ fromJson(steps.user_permission.outputs.data).permission }}' allowed values: 'admin', 'write'" | ||
| echo "::set-output name=allowed_user::true" | ||
|
|
||
| - name: Get information for pull request | ||
| if: github.event_name != 'workflow_dispatch' | ||
| uses: octokit/[email protected] | ||
| id: pr_api | ||
| with: | ||
| route: GET /repos/${{ github.repository }}/pulls/${{ github.event.issue.number }} | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Set outputs | ||
| id: set_outputs | ||
| run: | | ||
| set -eux | ||
|
|
||
| if [ ${{ github.event_name }} == 'workflow_dispatch' ]; then | ||
| echo "::set-output name=allowed_user::true" | ||
| echo "::set-output name=sha::$GITHUB_SHA" | ||
| else | ||
| echo "::set-output name=allowed_user::${{ steps.check_user_perm.outcome == 'success' && steps.check_user_perm.outputs.allowed_user }}" | ||
| echo "::set-output name=sha::${{ steps.pr_api.outcome == 'success' && fromJson(steps.pr_api.outputs.data).head.sha }}" | ||
| fi | ||
|
|
||
| outputs: | ||
| allowed_user: ${{ steps.set_outputs.outputs.allowed_user }} | ||
| sha: ${{ steps.set_outputs.outputs.sha }} | ||
|
|
||
| run: | ||
| needs: pr-info | ||
| # only do this for Fedora for now; once we have RHEL 8/9 boot.iso builds working, also support these | ||
| if: needs.pr-info.outputs.allowed_user == 'true' | ||
| runs-on: [self-hosted, kstest] | ||
| timeout-minutes: 300 | ||
| env: | ||
| STATUS_NAME: boot-iso | ||
| CONTAINER_TAG: 'lorax' | ||
| ISO_BUILD_CONTAINER_NAME: 'quay.io/rhinstaller/anaconda-iso-creator' | ||
| steps: | ||
| # we post statuses manually as this does not run from a pull_request event | ||
| # https://developer.github.com/v3/repos/statuses/#create-a-status | ||
| - name: Create in-progress status | ||
| uses: octokit/[email protected] | ||
| with: | ||
| route: 'POST /repos/${{ github.repository }}/statuses/${{ needs.pr-info.outputs.sha }}' | ||
| context: '${{ env.STATUS_NAME }} ${{ needs.pr-info.outputs.launch_args }}' | ||
| state: pending | ||
| target_url: 'https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}' | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Clone repository | ||
| uses: actions/checkout@v3 | ||
| with: | ||
| ref: ${{ needs.pr-info.outputs.sha }} | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Build anaconda-iso-creator container image | ||
| run: | | ||
| # set static tag to avoid complications when looking what tag is used | ||
| sudo make -f ./Makefile.am anaconda-iso-creator-build CI_TAG=$CONTAINER_TAG | ||
|
|
||
| - name: Build anaconda-rpm container (for RPM build) | ||
| run: | | ||
| # set static tag to avoid complications when looking what tag is used | ||
| make -f ./Makefile.am anaconda-rpm-build CI_TAG=$CONTAINER_TAG | ||
|
|
||
| - name: Build Anaconda RPM files | ||
| run: | | ||
| # output of the build will be stored in ./result/build/01-rpm-build/*.rpm | ||
| make -f ./Makefile.am container-rpms-scratch CI_TAG=$CONTAINER_TAG | ||
| mkdir -p ./anaconda_rpms/ | ||
| cp -av ./result/build/01-rpm-build/*.rpm ./anaconda_rpms/ | ||
|
|
||
| - name: Prepare environment for lorax run | ||
| run: | | ||
| mkdir -p images | ||
| # We have to pre-create loop devices because they are not namespaced in kernel so | ||
| # podman can't access newly created ones. That caused failures of tests when runners | ||
| # were rebooted. | ||
| sudo mknod -m 0660 /dev/loop0 b 7 0 2> /dev/null || true | ||
| sudo mknod -m 0660 /dev/loop1 b 7 1 2> /dev/null || true | ||
|
|
||
| - name: Build the boot.iso | ||
| run: | | ||
| # /var/tmp tmpfs speeds up lorax and avoids https://bugzilla.redhat.com/show_bug.cgi?id=1906364 | ||
| sudo podman run -i --rm --privileged \ | ||
| --tmpfs /var/tmp:rw,mode=1777 \ | ||
| -v `pwd`/anaconda_rpms:/anaconda-rpms:ro \ | ||
| -v `pwd`/images:/images:z \ | ||
| $ISO_BUILD_CONTAINER_NAME:$CONTAINER_TAG | ||
|
|
||
| - name: Collect logs | ||
| if: always() | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: 'logs' | ||
| path: | | ||
| images/*.log | ||
|
|
||
| - name: Upload image artifacts | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: images | ||
| path: | | ||
| images/boot.iso | ||
|
|
||
| - name: Set result status | ||
| if: always() | ||
| uses: octokit/[email protected] | ||
| with: | ||
| route: 'POST /repos/${{ github.repository }}/statuses/${{ needs.pr-info.outputs.sha }}' | ||
| context: '${{ env.STATUS_NAME }} ${{ needs.pr-info.outputs.launch_args }}' | ||
| state: ${{ job.status }} | ||
| target_url: 'https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}' | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| {% endif %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| {% if distro_release == "rawhide" %} | ||
| # a smaller sibling of the Fedora refresh; split to provide cleaner statuses | ||
| name: Refresh ELN container images | ||
| on: | ||
| schedule: | ||
| - cron: 0 0 * * * | ||
| # be able to start this action manually from a actions tab when needed | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
|
|
||
| eln: | ||
| uses: ./.github/workflows/container-rebuild-action.yml | ||
| secrets: inherit | ||
| with: | ||
| container-tag: eln | ||
| branch: master | ||
| base-container: 'quay.io/fedoraci/fedora:eln-x86_64' | ||
| {% endif %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| {% if distro_release == "rawhide" %} | ||
| name: Refresh Fedora container images | ||
| on: | ||
| schedule: | ||
| - cron: 0 0 * * * | ||
| # be able to start this action manually from a actions tab when needed | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
|
|
||
| master: | ||
| uses: ./.github/workflows/container-rebuild-action.yml | ||
| secrets: inherit | ||
| with: | ||
| container-tag: master | ||
| branch: master | ||
|
|
||
| {% if branched_fedora_version is defined and branched_fedora_version %} | ||
| f{$ branched_fedora_version $}-release: | ||
| uses: ./.github/workflows/container-rebuild-action.yml | ||
| secrets: inherit | ||
| with: | ||
| container-tag: f{$ branched_fedora_version $}-release | ||
| branch: f{$ branched_fedora_version $}-release | ||
| {% endif %} | ||
| {% endif %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.