Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): reusable worflow #387

Merged
merged 56 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
a76e54e
chore(ci): reusable worflow
magne4000 Sep 16, 2024
15bab7e
chore: ci
magne4000 Sep 16, 2024
d6663db
chore: ci
magne4000 Sep 16, 2024
2991fa6
chore: ci
magne4000 Sep 16, 2024
51d4ad3
chore: ci
magne4000 Sep 16, 2024
11be691
chore: ci
magne4000 Sep 16, 2024
ba0f6ee
chore: ci
magne4000 Sep 16, 2024
262a866
chore: ci
magne4000 Sep 16, 2024
0541287
chore: ci
magne4000 Sep 17, 2024
5cc213a
chore: add tests cli to split CI work
magne4000 Sep 17, 2024
a26df5a
chore: separate prepare test script
magne4000 Sep 17, 2024
e2ac806
chore: ci
magne4000 Sep 17, 2024
891fc39
chore: ci
magne4000 Sep 17, 2024
c9973ea
chore: ci
magne4000 Sep 17, 2024
487d78b
chore: ci
magne4000 Sep 17, 2024
592594a
chore: ci
magne4000 Sep 17, 2024
7ff0726
chore: ci
magne4000 Sep 17, 2024
99e0c08
chore: ci
magne4000 Sep 17, 2024
3ffd1fe
chore: ci
magne4000 Sep 17, 2024
18dddd7
chore: ci
magne4000 Sep 17, 2024
696ea07
chore: ci
magne4000 Sep 17, 2024
6323572
chore: ci
magne4000 Sep 17, 2024
4fef2fd
chore: ci
magne4000 Sep 17, 2024
daad334
chore: ci
magne4000 Sep 17, 2024
badb6f6
chore: ci
magne4000 Sep 17, 2024
2b7e6e0
chore: ci
magne4000 Sep 17, 2024
f45bf08
chore: ci
magne4000 Sep 17, 2024
94d3231
chore: ci
magne4000 Sep 17, 2024
4a94898
chore: ci
magne4000 Sep 17, 2024
4e7364d
chore: ci
magne4000 Sep 17, 2024
449221f
fix: do not rely on pnpm only for aws boilerplate
magne4000 Sep 17, 2024
8ef9056
fix: do not rely on pnpm only for aws boilerplate
magne4000 Sep 17, 2024
8ccd5bb
chore: ci
magne4000 Sep 17, 2024
59d6a42
chore: ci
magne4000 Sep 17, 2024
881cbb3
chore: ci
magne4000 Sep 17, 2024
1743a6d
fix: aws linting
magne4000 Sep 17, 2024
3bf254a
fix: aws linting
magne4000 Sep 17, 2024
c7520bd
chore: ci
magne4000 Sep 17, 2024
4902680
chore(ci): testing composite action
magne4000 Sep 18, 2024
d0abe6a
chore(ci): testing composite action
magne4000 Sep 18, 2024
37c73be
chore(ci): testing composite action
magne4000 Sep 18, 2024
28a7d3b
chore: ci
magne4000 Sep 18, 2024
9280a11
chore: ci
magne4000 Sep 18, 2024
d429843
chore: ci
magne4000 Sep 18, 2024
e63d832
chore: ci
magne4000 Sep 18, 2024
c3b9d0d
chore: ci
magne4000 Sep 18, 2024
8a15c4a
chore(ci): split load across a constrained amount of workers
magne4000 Sep 18, 2024
4e80047
chore: ci
magne4000 Sep 18, 2024
2c95c78
chore: ci
magne4000 Sep 18, 2024
083b636
chore(ci): split load across a constrained amount of workers
magne4000 Sep 18, 2024
6f76c16
chore: ci
magne4000 Sep 18, 2024
2332772
chore: ci
magne4000 Sep 18, 2024
a8af791
chore: ci
magne4000 Sep 18, 2024
e93a80f
chore: ci
magne4000 Sep 18, 2024
d902ac9
chore: ci
magne4000 Sep 18, 2024
c0febf6
chore: ci
magne4000 Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/actions/bati-run/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Step 5
name: Execute Bati CLI and run E2E tests

inputs:
os:
required: true
type: string
node:
required: true
default: 20
flags:
required: true
type: string
test-files:
required: true
type: string
destination:
required: true
type: string

runs:
using: "composite"

steps:
- name: Run Bati CLI
shell: bash
run: bun ./bati-cli/index.js ${{ inputs.flags }} ${{ inputs.destination }}

- name: Link tests-utils
shell: bash
run: cp -s ../bati-tests-utils/* .
working-directory: ${{ inputs.destination }}

- name: Link tests files
shell: bash
run: cp -s ../bati-tests-files/* .
working-directory: ${{ inputs.destination }}

- name: Prepare Bati tests
shell: bash
run: bun ../bati-tests/prepare.js --test-files='${{ inputs.test-files }}' ${{ inputs.flags }}
working-directory: ${{ inputs.destination }}

- name: Install dependencies
shell: bash
run: bun install
working-directory: ${{ inputs.destination }}

- name: Run build
shell: bash
run: bun turbo run build --no-update-notifier --framework-inference false --env-mode loose
working-directory: ${{ inputs.destination }}

- name: Run tests
shell: bash
run: bun turbo run test --only --no-update-notifier --framework-inference false --env-mode loose
working-directory: ${{ inputs.destination }}

- name: Run lint
shell: bash
run: bun turbo run lint --only --no-update-notifier --framework-inference false --env-mode loose
working-directory: ${{ inputs.destination }}

- name: Run typecheck
shell: bash
run: bun turbo run typecheck --only --no-update-notifier --framework-inference false --env-mode loose
working-directory: ${{ inputs.destination }}
168 changes: 168 additions & 0 deletions .github/workflows/tests-entry-os.reusable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Step 2
on:
workflow_call:
inputs:
os:
required: true
type: string
node:
required: true
type: number
fast:
type: boolean
jobs:
monorepo:
runs-on: ${{ inputs.os }}

# Debug discrepancies between published package and local one
# services:
# verdaccio:
# # See https://github.com/actions/runner/issues/822#issuecomment-1524826092
# image: ${{ (matrix.os == 'ubuntu-latest') && 'verdaccio/verdaccio' || '' }}
# ports:
# - 4873:4873

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node }}

- uses: pnpm/action-setup@v4
name: Install pnpm
id: pnpm-install
with:
run_install: false

- name: Get pnpm store directory
id: pnpm-cache
shell: bash
run: |
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV

- name: mkdir temp folder
run: mkdir -p ${{ runner.temp }}/bati-cache

- uses: actions/cache@v4
name: Setup pnpm cache
with:
path: ${{ env.STORE_PATH }}
key: ${{ inputs.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ inputs.os }}-pnpm-store-

- name: Cache for Turbo
uses: rharkor/[email protected]

# - name: Install global dependencies
# if: matrix.os == 'ubuntu-latest'
# run: pnpm install --global npm-cli-login

# - name: Login to verdaccio
# if: matrix.os == 'ubuntu-latest'
# run: npm-cli-login
# env:
# NPM_USER: bati
# NPM_PASS: bati
# NPM_EMAIL: [email protected]
# NPM_REGISTRY: http://localhost:4873

- name: Install dependencies
run: pnpm install

- name: Build packages
run: pnpm run build
env:
# self-contained Bati CLI, smaller artifact
BUNDLE: 1

# - name: Release in local repo
# if: matrix.os == 'ubuntu-latest'
# run: pnpm run release:local

- name: Run tests
run: pnpm run test
env:
BUNDLE: 1

- name: Deploy CLI locally
run: pnpm deploy --prod --filter=./packages/cli ./bati-cli

- name: Archive Bati CLI
uses: actions/upload-artifact@v4
with:
name: bati-cli-${{ inputs.os }}-${{ inputs.node }}
if-no-files-found: error
retention-days: 1
overwrite: true
include-hidden-files: true
path: ./bati-cli/dist/*

- name: Deploy tests locally
run: pnpm deploy --prod --filter=./packages/tests ./bati-tests

- name: Archive Bati tests
uses: actions/upload-artifact@v4
with:
name: bati-tests-${{ inputs.os }}-${{ inputs.node }}
if-no-files-found: error
retention-days: 1
overwrite: true
include-hidden-files: true
path: ./bati-tests/dist/*

- name: Pack tests-utils
run: pnpm pack
working-directory: ./packages/tests-utils

- name: Archive tests-utils
uses: actions/upload-artifact@v4
with:
name: tests-utils-${{ inputs.os }}-${{ inputs.node }}
if-no-files-found: error
retention-days: 1
overwrite: true
include-hidden-files: true
path: ./packages/tests-utils/batijs-tests-utils-*.tgz

- name: Archive tests files
uses: actions/upload-artifact@v4
with:
name: tests-files-${{ inputs.os }}-${{ inputs.node }}
if-no-files-found: error
retention-days: 1
overwrite: true
include-hidden-files: true
path: ./packages/tests/tests/*

- name: List e2e tests
id: list-e2e
run: pnpm run test:e2e list ${{ inputs.fast && '--workers 1 --filter=react,h3' || '--workers 5' }}
env:
TEST_AUTH0_CLIENT_ID: ${{ secrets.TEST_AUTH0_CLIENT_ID }}
TEST_AUTH0_ISSUER_BASE_URL: ${{ secrets.TEST_AUTH0_ISSUER_BASE_URL }}
TEST_FIREBASE_ACCOUNT: ${{ secrets.TEST_FIREBASE_ACCOUNT }}
TEST_FIREBASE_USER_UID: ${{ secrets.TEST_FIREBASE_USER_UID }}
TEST_GITHUB_CLIENT_ID: ${{ secrets.TEST_GITHUB_CLIENT_ID }}
TEST_GITHUB_CLIENT_SECRET: ${{ secrets.TEST_GITHUB_CLIENT_SECRET }}

outputs:
test-matrix: ${{ steps.list-e2e.outputs.test-matrix }}

generated:
needs: monorepo
strategy:
fail-fast: false
matrix:
test-matrix: ${{ fromJson(needs.monorepo.outputs.test-matrix) }}
name: "Trigger e2e tests: Group ${{ matrix.test-matrix[0] }}"

uses: ./.github/workflows/tests-generated.reusable.yml
secrets: inherit
with:
entry: ${{ matrix.test-matrix[1] }}
os: ${{ inputs.os }}
node: ${{ inputs.node }}
31 changes: 31 additions & 0 deletions .github/workflows/tests-entry.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Step 1
name: Tests

on:
pull_request:
paths-ignore:
- 'website/**'
push:
# Ensures cache is computed on main branch so that it can be reused on all PRs
branches: [ "main" ]
paths-ignore:
- 'website/**'

concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

jobs:
monorepo:
strategy:
matrix:
os: [ ubuntu-latest, macos-latest, windows-latest ]
node: [ 20 ]
fail-fast: false

uses: ./.github/workflows/tests-entry-os.reusable.yml
secrets: inherit
with:
os: ${{ matrix.os }}
node: ${{ matrix.node }}
fast: ${{ matrix.os != 'ubuntu-latest' }}
82 changes: 82 additions & 0 deletions .github/workflows/tests-generated.reusable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Step 3
on:
workflow_call:
inputs:
entry:
required: true
type: string
os:
required: true
type: string
node:
required: true
type: number
jobs:
reusable_workflow_job:
runs-on: ${{ inputs.os }}

env:
BATI_TEST: true
TEST_AUTH0_CLIENT_ID: ${{ secrets.TEST_AUTH0_CLIENT_ID }}
TEST_AUTH0_ISSUER_BASE_URL: ${{ secrets.TEST_AUTH0_ISSUER_BASE_URL }}
TEST_FIREBASE_ACCOUNT: ${{ secrets.TEST_FIREBASE_ACCOUNT }}
TEST_FIREBASE_USER_UID: ${{ secrets.TEST_FIREBASE_USER_UID }}
TEST_GITHUB_CLIENT_ID: ${{ secrets.TEST_GITHUB_CLIENT_ID }}
TEST_GITHUB_CLIENT_SECRET: ${{ secrets.TEST_GITHUB_CLIENT_SECRET }}

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node }}

- name: Cache for Turbo
uses: rharkor/[email protected]

- uses: oven-sh/setup-bun@v2
name: Install bun
with:
bun-version: "1.1.27"

- name: Download Bati CLI
uses: actions/download-artifact@v4
with:
name: bati-cli-${{ inputs.os }}-${{ inputs.node }}
path: ./bati-cli

- name: Download Bati tests
uses: actions/download-artifact@v4
with:
name: bati-tests-${{ inputs.os }}-${{ inputs.node }}
path: ./bati-tests

- name: Download tests-utils
uses: actions/download-artifact@v4
with:
name: tests-utils-${{ inputs.os }}-${{ inputs.node }}
path: ./bati-tests-utils

- name: Download tests-files
uses: actions/download-artifact@v4
with:
name: tests-files-${{ inputs.os }}-${{ inputs.node }}
path: ./bati-tests-files

- name: Generate dynamic composite action
run: node ./scripts/gen-composite-workflow-action.js '${{ inputs.entry }}'

# Step 4
- name: Run all tests
id: gen
uses: ./.github/actions/bati-gen
with:
os: ${{ inputs.os }}
node: ${{ inputs.node }}

- name: Cleanup
if: always() && steps.gen.outcome != 'success'
shell: bash
run: rm -rf ./.github/actions/bati-gen
Loading
Loading