Skip to content

Commit

Permalink
docs
Browse files Browse the repository at this point in the history
  • Loading branch information
julienrbrt committed Sep 4, 2024
1 parent 0377891 commit a1fab68
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 83 deletions.
15 changes: 13 additions & 2 deletions tests/systemtests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,32 @@
Test framework for system tests.
Starts and interacts with a (multi node) blockchain in Go.
Supports

* CLI
* Servers
* Events
* RPC

Uses:

* testify
* gjson
* sjson
Server and client side are executed on the host machine

Server and client side are executed on the host machine.

## Developer

### Test strategy

System tests cover the full stack via cli and a running (multi node) network. They are more expensive (in terms of time/ cpu)
to run compared to unit or integration tests.
Therefore, we focus on the **critical path** and do not cover every condition.

## How to use

Read the [getting_started.md](getting_started.md) guide to get started.

### Execute a single test

```sh
Expand All @@ -33,7 +42,9 @@ Test cli parameters
* `-nodes-count` int - number of nodes in the cluster (default 4)

# Port ranges

With *n* nodes:

* `26657` - `26657+n` - RPC
* `1317` - `1317+n` - API
* `9090` - `9090+n` - GRPC
Expand All @@ -47,4 +58,4 @@ For example Node *3* listens on `26660` for RPC calls

## Disclaimer

This is based on the system test framework in [wasmd](https://github.com/CosmWasm/wasmd) built by Confio.
This is based on the system test framework in [wasmd](https://github.com/CosmWasm/wasmd) built by Confio.
80 changes: 0 additions & 80 deletions tests/systemtests/bank_test.go

This file was deleted.

12 changes: 11 additions & 1 deletion tests/systemtests/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@

Build a new binary from current branch and copy it to the `tests/systemtests/binaries` folder by running system tests.
In project root:

```shell
make test-system
```

Or via manual steps

```shell
make build
mkdir -p ./tests/systemtests/binaries
Expand Down Expand Up @@ -39,6 +42,7 @@ func TestQueryTotalSupply(t *testing.T) {
t.Log("### got: " + raw)
}
```

The file begins with a Go build tag to exclude it from regular go test runs.
All tests in the `systemtests` folder build upon the *test runner* initialized in `main_test.go`.
This gives you a multi node chain started on your box.
Expand All @@ -57,6 +61,7 @@ go test -mod=readonly -tags='system_test' -v ./... --run TestQueryTotalSupply -

This give very verbose output. You would see all simd CLI commands used for starting the server or by the client to interact.
In the example code, we just log the output. Watch out for

```shell
bank_test.go:15: ### got: {
"supply": [
Expand Down Expand Up @@ -106,6 +111,7 @@ In order to test our assumptions in the system test, we modify the code to use `
assert.Equal(t, v, got, raw)
}
```

The assumption on the staking token usually fails due to inflation minted on the staking token. Let's fix this in the next step

### Run the test
Expand Down Expand Up @@ -158,6 +164,7 @@ of `gjson`, `sjson` and stdlib json operations.
})
sut.StartChain(t)
```

Next step is to add the new token to the assert map. But we can also make it more resilient to different node counts.

```go
Expand All @@ -183,8 +190,10 @@ The CLI wrapper works similar to the query. Just pass the parameters. It uses th
txHash := cli.Run("tx", "bank", "burn", "node0", "400000mytoken")
RequireTxSuccess(t, txHash)
```

`RequireTxSuccess` or `RequireTxFailure` can be used to ensure the expected result of the operation.
Next, check that the changes are applied.

```go
exp["mytoken"] = 600_000 // update expected state
raw = cli.CustomQuery("q", "bank", "total-supply")
Expand All @@ -197,9 +206,10 @@ Next, check that the changes are applied.

While tests are still more or less readable, it can gets harder the longer they are. I found it helpful to add
some comments at the beginning to describe what the intention is. For example:

```go
// scenario:
// given a chain with a custom token on genesis
// when an amount is burned
// then this is reflected in the total supply
```
```

0 comments on commit a1fab68

Please sign in to comment.