Skip to content

Commit

Permalink
WIP: Workflows
Browse files Browse the repository at this point in the history
  • Loading branch information
snaselj committed Oct 16, 2023
1 parent 5d957c9 commit 142bed0
Show file tree
Hide file tree
Showing 16 changed files with 670 additions and 36 deletions.
31 changes: 10 additions & 21 deletions .github/PULL_REQUEST_TEMPLATE/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,20 @@
<!--
Thank you for your interest in contributing to Nautobot! Please note
that our contribution policy recommends that a feature request or bug
report be opened for approval prior to filing a pull request. This
helps avoid wasting time and effort on something that we might not
be able to accept.
# Closes: NaN

Please indicate the relevant feature request or bug report below.
-->

# Closes: #<ISSUE NUMBER GOES HERE>
Optimize CI time.

## What's Changed

<!--
Please include:
- A summary of the proposed changes
- A sectioned breakdown for larger features under ## subheadings
- Screenshots, example payloads where relevant:
- Before/After for bugfixes
- Using a new feature
-->
- Added `build` GitHub action.
- Added simple `test-feature-pr.yml` GitHub workflow.
- Bumped CI `uses` versions to the latest major version each.
- Bumped CI image version to `ubuntu-22.04`.
- Disabled `ci.yml` GitHub workflow.
- Moved Compose dependencies to database `yaml` files.
- Added `--test-docs` to `invoke tests`.
- Fixed `invoke tests --lint-only` not to run coverage.

## To Do

<!--
Please feel free to update todos to keep track of your own notes for WIP PRs.
-->
- [ ] Explanation of Change(s)
- [ ] Added change log fragment(s) (for more information see [the documentation](https://docs.nautobot.com/projects/core/en/stable/development/#creating-changelog-fragments))
- [ ] Attached Screenshots, Payload Example
Expand Down
118 changes: 118 additions & 0 deletions .github/actions/docker-image/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
---
name: "Build"
description: "Build Docker Image for Nautobot App"
inputs:
image-prefix:
description: "Docker Image Prefix"
required: true
image-tag:
description: "Docker Image Tag"
required: true
nautobot-version:
description: "Nautobot Version"
required: true
password:
description: "GitHub Token"
required: false
python-version:
description: "Python Version"
required: true
username:
description: "GitHub Username"
required: false
push:
description: "Build and Push Docker Image"
required: false
default: ""
load:
description: "Build and Load Docker Image"
required: false
default: ""
pull:
description: "Pull Docker Image"
required: false
default: ""
runs:
using: "composite"
steps:
- name: "Configure"
shell: "bash"
id: "config"
env:
COMPOSE_FILE: "docker-compose.base.yml"
NAUTOBOT_VER: "${{ inputs.nautobot-version }}"
PYTHON_VER: "${{ inputs.python-version }}"
run: |
cd development
cp creds.example.env creds.env
# Read Docker image reference from Compose configuration
IMAGE="$(docker compose convert --format json | jq -r .services.nautobot.image)"
cd -
echo "image=${IMAGE}" | tee -a "$GITHUB_OUTPUT"
- name: Login to GitHub Container Registry
if: |
inputs.password != '' &&
inputs.username != ''
uses: docker/login-action@v3
with:
password: "${{ inputs.password }}"
registry: "ghcr.io"
username: "${{ inputs.username }}"
- name: "Set up Docker Buildx"
if: |
inputs.push == 'true' ||
inputs.load == 'true'
uses: "docker/setup-buildx-action@v3"
- name: "Build"
if: |
inputs.push == 'true'
uses: "docker/build-push-action@v5"
with:
push: true
context: "./"
file: "./development/Dockerfile"
tags: "${{ inputs.image-prefix }}:${{ inputs.image-tag }}"
cache-from: "type=gha,scope=${{ inputs.image-tag }}"
cache-to: "type=gha,scope=${{ inputs.image-tag }}"
build-args: |
NAUTOBOT_VER=${{ inputs.nautobot-version }}
PYTHON_VER=${{ inputs.python-version }}
- name: "Load"
if: |
inputs.load == 'true'
uses: "docker/build-push-action@v5"
with:
load: true
context: "./"
file: "./development/Dockerfile"
tags: "${{ inputs.image-prefix }}:${{ inputs.image-tag }}"
cache-from: "type=gha,scope=${{ inputs.image-tag }}"
build-args: |
NAUTOBOT_VER=${{ inputs.nautobot-version }}
PYTHON_VER=${{ inputs.python-version }}
- name: "Pull"
shell: "bash"
if: |
inputs.pull == 'true'
run: |
docker pull '${{ inputs.image-prefix }}:${{ inputs.image-tag }}'
- name: "Tag"
shell: "bash"
if: |
inputs.pull == 'true' || inputs.load == 'true'
run: |
docker tag '${{ inputs.image-prefix }}:${{ inputs.image-tag }}' '${{ steps.config.outputs.image }}'
outputs:
image-prefix:
description: "Docker Image Prefix"
value: "${{ inputs.image-prefix }}"
image-tag:
description: "Docker Image Tag"
value: "${{ inputs.image-tag }}"
ghcr-image:
description: "Docker Image Reference"
value: "${{ inputs.image-prefix }}:${{ inputs.image-tag }}"
compose-image:
description: "Docker Image Reference"
value: "${{ steps.config.outputs.image }}"
27 changes: 27 additions & 0 deletions .github/actions/run-linters/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
name: "Run Linters"
description: "Run Linters for Nautobot App"
inputs:
nautobot-version:
description: "Nautobot Version"
required: true
python-version:
description: "Python Version"
required: true
runs:
using: "composite"
steps:
- name: "Run Linters"
shell: "bash"
env:
COMPOSE_FILE: "docker-compose.base.yml:docker-compose.dev.yml"
NAUTOBOT_VER: "${{ inputs.nautobot-version }}"
PYTHON_VER: "${{ inputs.python-version }}"
run: |
cd development
docker-compose run \
--rm \
--entrypoint='' \
-- \
nautobot \
invoke tests --lint-only --no-test-docs
86 changes: 86 additions & 0 deletions .github/actions/unittests/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
name: "Unit Tests"
description: "Run Unit Tests for Nautobot App"
inputs:
db-backend:
description: "Database Backend"
required: true
nautobot-version:
description: "Nautobot Version"
required: true
python-version:
description: "Python Version"
required: true
runs:
using: "composite"
steps:
- name: "Configure"
id: "config"
shell: "bash"
run: |
export COMPOSE_FILE="docker-compose.base.yml:docker-compose.${{ inputs.db-backend }}.yml:docker-compose.redis.yml:docker-compose.dev.yml"
export NAUTOBOT_VER="${{ inputs.nautobot-version }}"
export PYTHON_VER="${{ inputs.python-version }}"
echo "COMPOSE_FILE=$COMPOSE_FILE" | tee -a "$GITHUB_ENV"
echo "NAUTOBOT_VER=$NAUTOBOT_VER" | tee -a "$GITHUB_ENV"
echo "PYTHON_VER=$PYTHON_VER" | tee -a "$GITHUB_ENV"
if [[ "${{ inputs.db-backend }}" == "mysql" ]]; then
cp invoke.mysql.yml invoke.yml
fi
cd development
docker compose up --detach -- db redis
DB_IMAGE="$(docker compose convert --format json | jq -r .services.db.image)"
CACHE_KEY="$(docker compose run --rm --entrypoint='' -- nautobot invoke calc-dbdump-cache-key --salt="$DB_IMAGE")"
echo "cache-key=$CACHE_KEY" | tee -a "$GITHUB_OUTPUT"
- name: "Cache Database Dump"
id: "cache"
uses: actions/cache@v3
with:
path: "development/dump.sql"
key: "${{ steps.config.outputs.cache-key }}"
restore-keys: "${{ steps.config.outputs.cache-key }}"
- name: "Use Cached Database Dump"
if: |
steps.cache.outputs.cache-hit == 'true'
shell: "bash"
run: |
cd development
docker compose exec -- db sh -c \
'${{ inputs.db-backend == 'mysql' &&
'mysql --user=nautobot --password=$MYSQL_PASSWORD' ||
'psql --username=nautobot postgres'
}}' \
< dump.sql
- name: "Build Database Dump"
if: |
steps.cache.outputs.cache-hit != 'true'
shell: "bash"
run: |
cd development
docker compose exec -- db sh -c \
'${{ inputs.db-backend == 'mysql' &&
'mysql -u nautobot -e "CREATE DATABASE test_nautobot;"' ||
'createdb --user=nautobot test_nautobot'
}}'
docker compose run --rm --entrypoint='' --env=NAUTOBOT_DB_NAME=test_nautobot -- nautobot \
nautobot-server migrate
docker compose exec -- db sh -c \
'${{ inputs.db-backend == 'mysql' &&
'mysqldump --user=root --password=$MYSQL_ROOT_PASSWORD --databases test_nautobot' ||
'pg_dump --clean --create --username=nautobot --dbname=test_nautobot'
}}' \
> dump.sql
- name: "Unit Tests"
shell: "bash"
run: |
cd development
docker compose run \
--rm \
--entrypoint='' \
-- \
nautobot \
invoke unittest --failfast --keepdb
41 changes: 41 additions & 0 deletions .github/workflows/build-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
name: "Build Image"
on: # yamllint disable-line rule:truthy
workflow_call:
inputs:
nautobot-version:
description: "Nautobot Version"
required: true
type: "string"
python-version:
description: "Python Version"
required: true
type: "string"
tag-prefix:
description: "Docker Image Tag Prefix"
required: true
type: "string"
username:
description: "GitHub Username"
required: true
type: "string"
secrets:
password:
description: "GitHub Token"
required: true
jobs:
build:
runs-on: "ubuntu-22.04"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v4"
- name: "Build and Push Docker Image"
uses: "./.github/actions/docker-image"
with:
image-prefix: "ghcr.io/${{ github.repository }}/nautobot-dev"
image-tag: "${{ inputs.tag-prefix }}-${{ inputs.nautobot-version }}-py${{ inputs.python-version }}"
nautobot-version: "${{ inputs.nautobot-version }}"
password: "${{ secrets.password }}"
push: true
python-version: "${{ inputs.python-version }}"
username: "${{ inputs.username }}"
16 changes: 9 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,15 @@ concurrency: # Cancel any existing runs of this workflow for this same PR
group: "${{ github.workflow }}-${{ github.ref }}"
cancel-in-progress: true
on: # yamllint disable-line rule:truthy rule:comments
push:
branches:
- "main"
- "develop"
tags:
- "v*"
pull_request: ~
# TODO: Temporary disabled
workflow_dispatch: {}
# push:
# branches:
# - "main"
# - "develop"
# tags:
# - "v*"
# pull_request: ~

env:
PLUGIN_NAME: "nautobot-plugin-firewall-models"
Expand Down
Loading

0 comments on commit 142bed0

Please sign in to comment.