Skip to content

Commit

Permalink
Merge branch 'refactor-workflows' into ref-dry-workflows
Browse files Browse the repository at this point in the history
  • Loading branch information
gustavovalverde authored Oct 17, 2023
2 parents a32b374 + 77b744d commit 1fc1422
Show file tree
Hide file tree
Showing 44 changed files with 1,130 additions and 453 deletions.
36 changes: 35 additions & 1 deletion .github/workflows/cd-deploy-nodes-gcp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,40 @@ jobs:
no_cache: ${{ inputs.no_cache || false }}
rust_log: info

# Test that Zebra works using the default config with the latest Zebra version.
test-configuration-file:
name: Test Zebra default Docker config file
needs: build
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'default-conf'
grep_patterns: '-e "net.*=.*Main.*estimated progress to chain tip.*BeforeOverwinter"'
test_variables: '-e NETWORK'
network: 'Mainnet'

# Test reconfiguring the the docker image for tesnet.
test-configuration-file-testnet:
name: Test Zebra testnet Docker config file
needs: build
# Make sure Zebra can sync the genesis block on testnet
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'testnet-conf'
grep_patterns: '-e "net.*=.*Test.*estimated progress to chain tip.*Genesis" -e "net.*=.*Test.*estimated progress to chain tip.*BeforeOverwinter"'
test_variables: '-e NETWORK -e ZEBRA_CONF_PATH="/etc/zebrad/zebrad.toml"'
network: 'Testnet'

# Test that Zebra works using $ZEBRA_CONF_PATH config
test-zebra-conf-path:
name: Test Zebra custom Docker config file
needs: build
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'custom-conf'
grep_patterns: '-e "v1.0.0-rc.2.toml"'
test_variables: '-e NETWORK -e ZEBRA_CONF_PATH="zebrad/tests/common/configs/v1.0.0-rc.2.toml"'
network: ${{ inputs.network || vars.ZCASH_NETWORK }}

# Deploy Managed Instance Groups (MiGs) for Mainnet and Testnet,
# with one node in the configured GCP region.
#
Expand All @@ -116,7 +150,7 @@ jobs:
matrix:
network: [Mainnet, Testnet]
name: Deploy ${{ matrix.network }} nodes
needs: [ build, versioning ]
needs: [ build, versioning, test-configuration-file, test-configuration-file-testnet, test-zebra-conf-path ]
runs-on: ubuntu-latest
timeout-minutes: 60
permissions:
Expand Down
139 changes: 25 additions & 114 deletions .github/workflows/ci-unit-tests-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -209,127 +209,38 @@ jobs:

# Test that Zebra works using the default config with the latest Zebra version.
test-configuration-file:
name: Test Zebra default Docker config file
timeout-minutes: 15
runs-on: ubuntu-latest
name: Test default config file
needs: build
steps:
- uses: r7kamura/[email protected]

- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7

- name: Run tests using the default config
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --detach --name default-conf-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} zebrad start
# Use a subshell to handle the broken pipe error gracefully
(
trap "" PIPE;
docker logs \
--tail all \
--follow \
default-conf-tests | \
tee --output-error=exit /dev/stderr | \
grep --max-count=1 --extended-regexp --color=always \
-e "net.*=.*Main.*estimated progress to chain tip.*BeforeOverwinter"
) || true
LOGS_EXIT_STATUS=$?
docker stop default-conf-tests
EXIT_STATUS=$(docker wait default-conf-tests || echo "Error retrieving exit status");
echo "docker exit status: $EXIT_STATUS";
# If grep found the pattern, exit with the Docker container exit status
if [ $LOGS_EXIT_STATUS -eq 0 ]; then
exit $EXIT_STATUS;
fi
# Handle other potential errors here
echo "An error occurred while processing the logs.";
exit 1;
env:
NETWORK: Mainnet
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'default-conf'
grep_patterns: '-e "net.*=.*Main.*estimated progress to chain tip.*BeforeOverwinter"'
test_variables: '-e NETWORK'
network: 'Mainnet'

# Test reconfiguring the docker image for testnet.
# Test reconfiguring the the docker image for tesnet.
test-configuration-file-testnet:
name: Test Zebra testnet Docker config file
timeout-minutes: 15
runs-on: ubuntu-latest
name: Test testnet config file
needs: build
steps:
- uses: r7kamura/[email protected]

- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7

# Make sure Zebra can sync the genesis block on testnet
- name: Run tests using a testnet config
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK -e ZEBRA_CONF_PATH -e ENTRYPOINT_FEATURES='' --detach --name testnet-conf-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} zebrad start
# Use a subshell to handle the broken pipe error gracefully
(
trap "" PIPE;
docker logs \
--tail all \
--follow \
testnet-conf-tests | \
tee --output-error=exit /dev/stderr | \
grep --max-count=1 --extended-regexp --color=always \
-e "net.*=.*Test.*estimated progress to chain tip.*Genesis" \
-e "net.*=.*Test.*estimated progress to chain tip.*BeforeOverwinter";
) || true
LOGS_EXIT_STATUS=$?
docker stop testnet-conf-tests
EXIT_STATUS=$(docker wait testnet-conf-tests || echo "Error retrieving exit status");
echo "docker exit status: $EXIT_STATUS";
# If grep found the pattern, exit with the Docker container exit status
if [ $LOGS_EXIT_STATUS -eq 0 ]; then
exit $EXIT_STATUS;
fi
# Handle other potential errors here
echo "An error occurred while processing the logs.";
exit 1;
env:
ZEBRA_CONF_PATH: '/etc/zebrad/zebrad.toml'
NETWORK: Testnet
# Make sure Zebra can sync the genesis block on testnet
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'testnet-conf'
grep_patterns: '-e "net.*=.*Test.*estimated progress to chain tip.*Genesis" -e "net.*=.*Test.*estimated progress to chain tip.*BeforeOverwinter"'
# TODO: improve the entrypoint to avoid using `ENTRYPOINT_FEATURES=""`
test_variables: '-e NETWORK -e ZEBRA_CONF_PATH="/etc/zebrad/zebrad.toml" -e ENTRYPOINT_FEATURES=""'
network: 'Testnet'

# Test that Zebra works using the $ZEBRA_CONF_PATH config
# Test that Zebra works using $ZEBRA_CONF_PATH config
test-zebra-conf-path:
name: Test Zebra custom Docker config file
timeout-minutes: 15
runs-on: ubuntu-latest
name: Test custom config file
needs: build
steps:
- uses: r7kamura/[email protected]

- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7

- name: Run tests using the $ZEBRA_CONF_PATH
run: |
set -ex
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --detach -e ZEBRA_CONF_PATH --name variable-conf-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} zebrad start
EXIT_STATUS=$(docker logs --tail all --follow variable-conf-tests 2>&1 | grep -q --extended-regexp --max-count=1 -e 'v1.0.0-rc.2.toml'; echo $?; )
docker stop variable-conf-tests
docker logs variable-conf-tests
exit "$EXIT_STATUS"
env:
ZEBRA_CONF_PATH: 'zebrad/tests/common/configs/v1.0.0-rc.2.toml'
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'custom-conf'
grep_patterns: '-e "v1.0.0-rc.2.toml"'
test_variables: '-e NETWORK -e ZEBRA_CONF_PATH="zebrad/tests/common/configs/v1.0.0-rc.2.toml"'
network: ${{ inputs.network || vars.ZCASH_NETWORK }}

failure-issue:
name: Open or update issues for main branch failures
Expand Down
75 changes: 75 additions & 0 deletions .github/workflows/sub-test-zebra-config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Test Zebra Config Files

on:
workflow_call:
inputs:
# Status and logging
test_id:
required: true
type: string
description: 'Unique identifier for the test'
grep_patterns:
required: true
type: string
description: 'Patterns to grep for in the logs'

# Test selection and parameters
test_variables:
required: true
type: string
description: 'Environmental variables used to select and configure the test'
network:
required: false
type: string
default: Mainnet
description: 'Zcash network to test against'

jobs:
test-docker-config:
name: Test ${{ inputs.test_id }} in Docker
timeout-minutes: 15
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
with:
persist-credentials: false

- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7

- uses: r7kamura/[email protected]

- name: Run ${{ inputs.test_id }} test
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run ${{ inputs.test_variables }} --detach --name ${{ inputs.test_id }} -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} zebrad start
# Use a subshell to handle the broken pipe error gracefully
(
trap "" PIPE;
docker logs \
--tail all \
--follow \
${{ inputs.test_id }} | \
tee --output-error=exit /dev/stderr | \
grep --max-count=1 --extended-regexp --color=always \
${{ inputs.grep_patterns }}
) || true
LOGS_EXIT_STATUS=$?
docker stop ${{ inputs.test_id }}
EXIT_STATUS=$(docker wait ${{ inputs.test_id }} || echo "Error retrieving exit status");
echo "docker exit status: $EXIT_STATUS";
# If grep found the pattern, exit with the Docker container exit status
if [ $LOGS_EXIT_STATUS -eq 0 ]; then
exit $EXIT_STATUS;
fi
# Handle other potential errors here
echo "An error occurred while processing the logs.";
exit 1;
env:
NETWORK: '${{ inputs.network }}'
60 changes: 60 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,66 @@ All notable changes to Zebra are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org).

## [Zebra 1.3.0](https://github.com/ZcashFoundation/zebra/releases/tag/v1.3.0) - 2023-10-16

This release adds RPC methods for the "Spend before Sync" light wallet feature,
and fixes performance issues and bugs in the mining solution rate RPCs. Progress
bars can now be enabled using a config, please help us test them!

It contains the following updates:

### User Testing: Progress Bars

Zebra has progress bars! When progress bars are enabled, you can see Zebra's blocks,
transactions, and peer connections in your terminal. We're asking Zebra users to test this
feature, and give us [feedback on the forums](https://forum.zcashcommunity.com/t/zebra-progress-bars/44485).

To show progress bars while running Zebra, add these lines to your `zebrad.toml`:
```toml
[tracing]
progress_bar = "summary"
```

For more details, including a known issue with time estimates,
read our [progress bars blog post](https://zfnd.org/experimental-zebra-progress-bars/).

### Security

- Fix database concurrency bugs that could have led to panics or incorrect history tree data (#7590, #7663)

### Added

- Zebra's progress bars can now be enabled using a `zebrad.toml` config (#7615)
- Add missing elasticsearch flag feature to lib docs (#7568)
- Add missing Docker variables and examples (#7552)
- Check database format is valid on startup and shutdown (#7566, #7606). We expect to catch almost all database validity errors in CI (#7602, #7627), so users are unlikely to see them on startup or shutdown.

#### Spend before Sync Support

- Add state requests and support code for the `z_getsubtreesbyindex` RPC (#7408, #7734)
- Implement the `z_getsubtreesbyindex` RPC (#7436)
- Test the `z_getsubtreesbyindex` RPC (#7515, #7521, #7566, #7514, #7628)
- Format subtree roots in little-endian order (#7466)
- Add note subtree indexes for new and existing blocks (#7437)
- Upgrade subtrees from the tip backwards, for compatibility with wallet syncing (#7531)
- Handle a subtree comparison edge case correctly (#7587)

### Changed

- Return errors instead of panicking in methods for Heights (#7591)
- Update tests for compatibility with the ECC's `lightwalletd` fork (#7349)

### Fixed

- Refactor docs for feature flags (#7567)
- Match zcashd's getblockchaininfo capitalisation for NU5 (#7454)
- Fix bugs and performance of `getnetworksolps` & `getnetworkhashps` RPCs (#7647)

### Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @oxarbitrage, @rex4539, @teor2345 and @upbqdn.

## [Zebra 1.2.0](https://github.com/ZcashFoundation/zebra/releases/tag/v1.2.0) - 2023-09-01

### Highlights
Expand Down
Loading

0 comments on commit 1fc1422

Please sign in to comment.