Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel release test #2239

Closed
wants to merge 61 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
3b6b84c
try parallel release
bdemann Oct 14, 2024
b6aef35
WIP
bdemann Oct 14, 2024
2024b93
move into workflow
bdemann Oct 14, 2024
f78d288
fixup
bdemann Oct 14, 2024
52d6840
pass npm token secret
bdemann Oct 15, 2024
bff3610
fixup
bdemann Oct 15, 2024
a391563
fixup
bdemann Oct 15, 2024
1d4468d
install dfx before npm install
bdemann Oct 15, 2024
042cb70
make sure all commits are verified
bdemann Oct 15, 2024
1d94df3
add release version to the branch names
bdemann Oct 15, 2024
6a91528
add env variable
bdemann Oct 15, 2024
6bab4a4
update checkout
bdemann Oct 15, 2024
9b2f482
create branch
bdemann Oct 15, 2024
2da525f
remove sub release trouble shooter
bdemann Oct 15, 2024
41cffb9
make sure branches are unique
bdemann Oct 15, 2024
76f1f52
try these things
bdemann Oct 15, 2024
c539b67
update finalize release step
bdemann Oct 15, 2024
14fcc97
fetch the branch
bdemann Oct 15, 2024
d85a32d
clean up
bdemann Oct 16, 2024
58ba02b
use PAT
bdemann Oct 16, 2024
fcef3f5
correctly bring in the lastmjs pat
bdemann Oct 18, 2024
d4ebd9b
remove explicit path
bdemann Oct 18, 2024
f750d1a
Merge branch 'main' into parallel_release
lastmjs Oct 22, 2024
1b7da25
pr fixes
bdemann Oct 23, 2024
bda1f3c
further update names
bdemann Oct 23, 2024
73d0126
restore old husky install
bdemann Oct 24, 2024
10c660f
get rid of GH_TOKEN
bdemann Oct 24, 2024
e223122
Merge branch 'main' into parallel_release
bdemann Oct 24, 2024
eee67f6
install azle before updating azle
bdemann Oct 24, 2024
6386be2
bring back GH_TOKEN
bdemann Oct 24, 2024
ec0424d
try just fetching as we need
bdemann Oct 24, 2024
df84c5d
simpler getting of the current branch
bdemann Oct 24, 2024
39f42da
try a squashing strategy
bdemann Oct 24, 2024
bd559c3
just do fetch all
bdemann Oct 24, 2024
9929a89
switch to cherry picking
bdemann Oct 24, 2024
b81f015
do a soft reset to squash all commits into one
bdemann Oct 24, 2024
622de9d
try squash again
bdemann Oct 24, 2024
14df4fd
move install dfx into a setup dfx action
bdemann Oct 25, 2024
9cb3b93
fix include_npm
bdemann Oct 25, 2024
790eccb
run tests for benchmarks during release
bdemann Oct 25, 2024
d24b466
clean up fetching workflow
bdemann Oct 25, 2024
8980ae1
batch delete branches
bdemann Oct 25, 2024
406b869
batch fetching
bdemann Oct 25, 2024
5549518
make delete like fetch
bdemann Oct 25, 2024
b778706
simplification
bdemann Oct 28, 2024
aa3beb1
better display
bdemann Oct 28, 2024
a5c85b2
switch to while read to
bdemann Oct 28, 2024
c744d26
clean up counts and expand run
bdemann Oct 28, 2024
68cf4a2
don't use pipe so that the subshell won't eat the output
bdemann Oct 28, 2024
37fcbcf
create configure git action
bdemann Oct 28, 2024
7a3585e
remove install curl
bdemann Oct 28, 2024
d1bf022
create setup-node action
bdemann Oct 28, 2024
308968e
clean up code
bdemann Oct 28, 2024
5c9b025
remove inputs from set-up node
bdemann Oct 28, 2024
e800a20
get rid or dfx input
bdemann Oct 28, 2024
c15b223
update branch name
bdemann Oct 28, 2024
e13123e
delete file
bdemann Oct 28, 2024
7871dfa
bring back the repo
bdemann Oct 28, 2024
26ee9ce
try this
bdemann Oct 28, 2024
fb0b14e
try all of this
bdemann Oct 28, 2024
886b712
try that
bdemann Oct 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/actions/configure_git/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
## Prerequisite

This action assumes that the repository has already been checked out before
calling the action, typically using `actions/checkout@v4`. If you have not
checked out the code in a previous step, make sure to do so to avoid errors.

This action does **not** perform a checkout action itself because it would be
redundant. This action is part of the repository's codebase, so if the code
hasn't already been checked out, the action itself wouldn't even be available to
call.

## Example Usage

```yaml
steps:
- uses: actions/checkout@v4

- uses: ./.github/actions/configure_git
with:
gpg_signing_key: ${{ secrets.GPG_SIGNING_KEY }}
```
18 changes: 18 additions & 0 deletions .github/actions/configure_git/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Configure Git
description: 'Configures git with user info and GPG signing'
inputs:
gpg_signing_key:
description: 'The GPG signing key to use for signing commits'
required: true
runs:
using: composite
steps:
- name: Configure git
run: |
# TODO we should use some Action-specific bot account
git config --global user.name 'Jordan Last'
git config --global user.email '[email protected]'
git config --global commit.gpgsign true
echo -n "${{ inputs.gpg_signing_key }}" | base64 --decode | gpg --import
git config --global user.signingkey C8B77BCBE16CD2B94B43F9C8757397B82D4ED7B0
shell: bash
14 changes: 0 additions & 14 deletions .github/actions/get_dfx_version/action.yml

This file was deleted.

14 changes: 0 additions & 14 deletions .github/actions/get_node_version/action.yml

This file was deleted.

13 changes: 7 additions & 6 deletions .github/actions/get_test_infos/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,7 @@ outputs:
runs:
using: composite
steps:
- id: get-node-version
uses: ./.github/actions/get_node_version

- uses: actions/setup-node@v4
with:
node-version: ${{ steps.get-node-version.outputs.node-version }}
- uses: ./.github/actions/setup_node

- name: Get test infos
id: get-test-infos
Expand All @@ -40,3 +35,9 @@ runs:
TEST_INFOS=$(./.github/actions/get_test_infos/get_test_infos.sh | base64 -d)
echo "test-infos=${TEST_INFOS}" >> "$GITHUB_OUTPUT"
shell: bash

- name: Echo test infos for troubleshooting
run: |
echo "Test Infos Output:"
echo "${{ steps.get-test-infos.outputs.test-infos }}"
shell: bash
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ checking out a specific branch.
steps:
- uses: actions/checkout@v4

- id: get-dfx-version
uses: ./.github/actions/get_dfx_version
- id: setup-dfx
uses: ./.github/actions/setup_dfx

- run: echo ${{ steps.get-dfx-version.outputs.dfx-version }}
- run: echo ${{ steps.setup-dfx.outputs.dfx-version }}
```
21 changes: 21 additions & 0 deletions .github/actions/setup_dfx/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Setup dfx
description: 'Sets up dfx by detecting version from package.json and installing it. (Note: DFX must be installed before `npm install` because the azle installation process requires dfx)'
outputs:
dfx-version:
description: 'The version of dfx that was installed'
value: ${{ steps.get-dfx-version.outputs.dfx-version }}
runs:
using: composite
steps:
- id: get-dfx-version
run: |
DFX_VERSION=$(jq -r '.azle.globalDependencies.dfx // error("dfx version not found")' "package.json")
echo "dfx-version=${DFX_VERSION}" >> "$GITHUB_OUTPUT"
shell: bash

- name: Install dfx
run: |
# Install dfx (Note: dfx must be installed before `npx azle` because the azle installation process requires dfx)
src/build/stable/commands/install_global_dependencies/install_dfx.sh "${{ steps.get-dfx-version.outputs.dfx-version }}"
echo "$HOME/.local/share/dfx/bin" >> $GITHUB_PATH
shell: bash
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ checking out a specific branch.
steps:
- uses: actions/checkout@v4

- id: get-node-version
uses: ./.github/actions/get_node_version
- id: setup-node
uses: ./.github/actions/setup_node

- run: echo "${{ steps.get-node-version.outputs.node-version }}"
- run: echo "${{ steps.setup-node.outputs.version }}"
```
29 changes: 29 additions & 0 deletions .github/actions/setup_node/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: 'Setup Node.js'
description: 'Sets up Node.js environment and gets Node version from package.json'
inputs:
registry-url:
description: 'Optional registry URL'
required: false
node-version-file:
description: 'File containing the version Spec'
required: false
default: 'package.json'
outputs:
node-version:
description: 'The Node.js version from package.json'
value: ${{ steps.get-version.outputs.version }}

runs:
using: 'composite'
steps:
- id: get-version
shell: bash
run: |
VERSION=$(jq -r '.azle.globalDependencies.node // error("node version not found")' "package.json")
echo "version=$VERSION" >> $GITHUB_OUTPUT

- uses: actions/setup-node@v4
with:
registry-url: ${{ inputs.registry-url }}
node-version: ${{ steps.get-version.outputs.version }}
node-version-file: ${{ inputs.node-version-file }}
59 changes: 0 additions & 59 deletions .github/scripts/publish_github_action.sh

This file was deleted.

84 changes: 9 additions & 75 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ on:
push:
branches:
- main
pull_request: # Runs on pull requests to any branch
pull_request:

jobs:
determine-should-release:
if: ${{ startsWith(github.head_ref, 'release--') }}
Expand All @@ -17,84 +18,17 @@ jobs:
- id: determine-should-release
uses: ./.github/actions/should_release

get-test-infos:
needs: determine-should-release
if: ${{ startsWith(github.head_ref, 'release--') && needs.determine-should-release.outputs.should-release }}
name: Get test infos
runs-on: ubuntu-latest
outputs:
test-infos: ${{ steps.get-test-infos.outputs.test-infos }}
steps:
- uses: actions/checkout@v4

- id: get-node-version
uses: ./.github/actions/get_node_version

- name: Get all test infos
id: get-test-infos
uses: ./.github/actions/get_test_infos
with:
node-version: ${{ steps.get-node-version.outputs.node-version }}
directories: |
./examples
./tests

release:
name: Deploy release
# Only run this job if it's a release branch. This job will run instead of run-tests and will automatically publish another commit which will be tested
if: ${{ needs.determine-should-release.outputs.should-release == 'true' }}

needs:
- determine-should-release
- get-test-infos
runs-on: ubuntu-latest
env:
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }} # All commits must be verified
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.ref || github.ref }}
token: ${{ secrets.LASTMJS_GITHUB_TOKEN || github.token }}

- id: get-node-version
uses: ./.github/actions/get_node_version

- uses: actions/setup-node@v4
with:
node-version: ${{ steps.get-node-version.outputs.node-version }}
registry-url: https://registry.npmjs.org
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

- name: Install curl
run: sudo apt-get install curl -y

- id: get-dfx-version
uses: ./.github/actions/get_dfx_version

- name: Install dfx
run: |
# Install dfx (Note: dfx must be installed before `npx azle` because the azle installation process requires dfx)
src/build/stable/commands/install_global_dependencies/install_dfx.sh ${{ steps.get-dfx-version.outputs.dfx-version }}
echo "$HOME/.local/share/dfx/bin" >> $GITHUB_PATH

- run: npm install

- name: Install global dependencies
run: |
AZLE_VERBOSE=true npx azle install-global-dependencies --rust --wasi2ic

# TODO we should use some Action-specific bot account
- name: Configure git for publishing release
run: |
git config --global user.name 'Jordan Last'
git config --global user.email '[email protected]'
git config --global commit.gpgsign true
echo -n "$GPG_SIGNING_KEY" | base64 --decode | gpg --import
git config --global user.signingkey C8B77BCBE16CD2B94B43F9C8757397B82D4ED7B0

- name: Publish release
run: |
BRANCH_NAME="${{ github.head_ref }}"
RELEASE_VERSION="${BRANCH_NAME:9}"
./.github/scripts/publish_github_action.sh $RELEASE_VERSION ${{ toJSON(needs.get-test-infos.outputs.test-infos) }}
uses: ./.github/workflows/release_parallel.yml
secrets:
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
LASTMJS_GITHUB_TOKEN: ${{ secrets.LASTMJS_GITHUB_TOKEN }}
Loading
Loading