Skip to content

Commit

Permalink
Merge pull request #47 from ava-labs/consolidate-e2e
Browse files Browse the repository at this point in the history
Consolidate e2e
  • Loading branch information
cam-schultz authored Oct 4, 2023
2 parents fc35ffa + c40f0e4 commit 8908f1d
Show file tree
Hide file tree
Showing 12 changed files with 419 additions and 660 deletions.
21 changes: 3 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,6 @@ The relayer consists of the following components:

## Testing

---

### Unit tests

Unit tests can be ran locally by running the command in root of the project:
Expand All @@ -61,25 +59,12 @@ Unit tests can be ran locally by running the command in root of the project:

### E2E tests

E2E tests are ran as part of CI, but can also be ran locally. To run the E2E tests locally, you'll need to do the following:
- Install Gingko following the intructions [here](https://onsi.github.io/ginkgo/#installing-ginkgo)
- Clone `subnet-evm` from [Github](https://github.com/ava-labs/subnet-evm) and checkout the correct version as specified in `go.mod` or `scripts/versions.sh`

Next, set up the `avalanchego` build path and `subnet-evm` binary, making sure to install everything in a writeable location (here we use `~/tmp`):
E2E tests are ran as part of CI, but can also be ran locally with the `--local` flag. To run the E2E tests locally, you'll need to install Gingko following the intructions [here](https://onsi.github.io/ginkgo/#installing-ginkgo)

Next, provide the path to the `subnet-evm` repository and the path to a writeable data directory (here we use the `~/subnet-evm` and `~/tmp/e2e-test`) to use for the tests:
```bash
cd subnet-evm
BASEDIR=~/tmp/e2e-test AVALANCHEGO_BUILD_PATH=~/tmp/e2e-test/avalanchego ./scripts/install_avalanchego_release.sh
./scripts/build.sh ~/tmp/e2e-test/avalanchego/plugins/srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy
./scripts/e2e_test.sh --local --subnet-evm ~/subnet-evm --data-dir ~/tmp/e2e-test
```

Then, in the root of the `awm-relayer` project, run:

```bash
AVALANCHEGO_BUILD_PATH=~/tmp/e2e-test/avalanchego DATA_DIR=~/tmp/e2e-test/data ./scripts/e2e_test.sh
```

Note that any additional E2E tests that run VMs other than `subnet-evm` will need to install and setup the VM binary in the same way.
### Generate Mocks

We use [gomock](https://pkg.go.dev/go.uber.org/mock/gomock) to generate mocks for testing. To generate mocks, run the following command at the root of the project:
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
github.com/ava-labs/avalanche-network-runner v1.7.2
github.com/ava-labs/avalanchego v1.10.10
github.com/ava-labs/subnet-evm v0.5.6
github.com/ava-labs/teleporter v0.0.0-20231004221622-b655bfe85981
github.com/ethereum/go-ethereum v1.12.0
github.com/onsi/ginkgo/v2 v2.12.0
github.com/onsi/gomega v1.27.10
Expand Down
6 changes: 6 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,12 @@ github.com/ava-labs/coreth v0.12.5-rc.6 h1:OajGUyKkO5Q82XSuMa8T5UD6QywtCHUiZ4Tv3
github.com/ava-labs/coreth v0.12.5-rc.6/go.mod h1:s5wVyy+5UCCk2m0Tq3jVmy0UqOpKBDYqRE13gInCJVs=
github.com/ava-labs/subnet-evm v0.5.6 h1:u+xBvEExOa362Up02hgSgeF+aqDona57whhRIeEIim8=
github.com/ava-labs/subnet-evm v0.5.6/go.mod h1:desGY3ghT+Ner+oqxeovwF37eM4dmMQbYZECONPQU9w=
github.com/ava-labs/teleporter v0.0.0-20231002210825-d5f6d7f8583a h1:KOj9SYdVCK736YyklPqBgOrlMP7JfN5L88mOl2dPj88=
github.com/ava-labs/teleporter v0.0.0-20231002210825-d5f6d7f8583a/go.mod h1:nyjuYCBefAGCkKDhmJIxh8iTAczapQxwxj/7FC4g9sU=
github.com/ava-labs/teleporter v0.0.0-20231004214459-caa08b68d3b4 h1:1gYjG8pi1EnHBZbuXPZKwlzZ6UztlCcaif9o9+CG6K0=
github.com/ava-labs/teleporter v0.0.0-20231004214459-caa08b68d3b4/go.mod h1:nyjuYCBefAGCkKDhmJIxh8iTAczapQxwxj/7FC4g9sU=
github.com/ava-labs/teleporter v0.0.0-20231004221622-b655bfe85981 h1:beCzXayPAc9obFYmPyF7WOfyPCIYWThUIB9Jcxbtgq0=
github.com/ava-labs/teleporter v0.0.0-20231004221622-b655bfe85981/go.mod h1:nyjuYCBefAGCkKDhmJIxh8iTAczapQxwxj/7FC4g9sU=
github.com/aymerick/raymond v2.0.3-0.20180322193309-b565731e1464+incompatible/go.mod h1:osfaiScAUVup+UC9Nfq76eWqDhXlp+4UYaA8uhTBO6g=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
Expand Down
45 changes: 45 additions & 0 deletions scripts/e2e_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,51 @@

set -e

SUBNET_EVM_PATH=
LOCAL=
DATA_DIRECTORY=
HELP=
while [ $# -gt 0 ]; do
case "$1" in
-l | --local) LOCAL=true ;;
-s | --subnet-evm) SUBNET_EVM_PATH=$2 ;;
-d | --data-dir) DATA_DIRECTORY=$2 ;;
-h | --help) HELP=true ;;
esac
shift
done

if [ "$HELP" = true ]; then
echo "Usage: ./scripts/e2e_test.sh [OPTIONS]"
echo "Run E2E tests for AWM Relayer."
echo ""
echo "Options:"
echo " -l, --local Run the test locally. Requires --subnet-evm and --data-dir"
echo " -s, --subnet-evm <path> Path to subnet-evm repo"
echo " -d, --data-dir <path> Path to data directory"
echo " -h, --help Print this help message"
exit 0
fi

if [ "$LOCAL" = true ]; then
if [ -z "$DATA_DIRECTORY" ]; then
echo "Must specify data directory when running local"
exit 1
fi
if [ -z "$SUBNET_EVM_PATH" ]; then
echo "Must specify subnet-evm path when running local"
exit 1
fi
cwd=$PWD
cd $SUBNET_EVM_PATH
BASEDIR=$DATA_DIRECTORY AVALANCHEGO_BUILD_PATH=$DATA_DIRECTORY/avalanchego ./scripts/install_avalanchego_release.sh
./scripts/build.sh $DATA_DIRECTORY/avalanchego/plugins/srEXiWaHuhNyGwPUi444Tu47ZEDwxTWrbQiuD7FmgSAQ6X7Dy

cd $cwd
export AVALANCHEGO_BUILD_PATH=$DATA_DIRECTORY/avalanchego
export DATA_DIR=$DATA_DIRECTORY/data
fi

RELAYER_PATH=$(
cd "$(dirname "${BASH_SOURCE[0]}")"
cd .. && pwd
Expand Down
287 changes: 287 additions & 0 deletions tests/basic_relay.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,287 @@
package tests

import (
"context"
"encoding/hex"
"encoding/json"
"fmt"
"math/big"
"os"
"os/exec"
"time"

"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/avalanchego/utils/logging"
avalancheWarp "github.com/ava-labs/avalanchego/vms/platformvm/warp"
"github.com/ava-labs/awm-relayer/config"
"github.com/ava-labs/awm-relayer/database"
"github.com/ava-labs/awm-relayer/messages/teleporter"
"github.com/ava-labs/awm-relayer/peers"
testUtils "github.com/ava-labs/awm-relayer/tests/utils"
"github.com/ava-labs/subnet-evm/core/types"
predicateutils "github.com/ava-labs/subnet-evm/utils/predicate"
warpPayload "github.com/ava-labs/subnet-evm/warp/payload"
"github.com/ava-labs/subnet-evm/x/warp"
teleportermessenger "github.com/ava-labs/teleporter/abis/TeleporterMessenger"
teleporterTestUtils "github.com/ava-labs/teleporter/tests/utils"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/log"
. "github.com/onsi/gomega"
)

var (
storageLocation = fmt.Sprintf("%s/.awm-relayer-storage", os.TempDir())
)

// Ginkgo describe node that acts as a container for the relayer e2e tests. This test suite
// will run in order, starting off by setting up the subnet URIs and creating a relayer config
// file. It will then build the relayer binary and run it with the config file. The relayer
// will then send a transaction to the source subnet to issue a Warp message simulting a transaction
// sent from the Teleporter contract. The relayer will then wait for the transaction to be confirmed
// on the destination subnet and verify that the Warp message was received and unpacked correctly.
func BasicRelay() {
var (
receivedWarpMessage *avalancheWarp.Message
payload []byte
relayerCmd *exec.Cmd
relayerCancel context.CancelFunc
)

subnetAInfo := teleporterTestUtils.GetSubnetATestInfo()
subnetBInfo := teleporterTestUtils.GetSubnetBTestInfo()
teleporterContractAddress := teleporterTestUtils.GetTeleporterContractAddress()
fundedAddress, fundedKey := teleporterTestUtils.GetFundedAccountInfo()

teleporterMessage := teleporter.TeleporterMessage{
MessageID: big.NewInt(1),
SenderAddress: fundedAddress,
DestinationAddress: fundedAddress,
RequiredGasLimit: big.NewInt(1),
AllowedRelayerAddresses: []common.Address{},
Receipts: []teleporter.TeleporterMessageReceipt{},
Message: []byte{1, 2, 3, 4},
}

//
// Set up relayer config
//
hostA, portA, err := teleporterTestUtils.GetURIHostAndPort(subnetAInfo.ChainNodeURIs[0])
Expect(err).Should(BeNil())

hostB, portB, err := teleporterTestUtils.GetURIHostAndPort(subnetBInfo.ChainNodeURIs[0])
Expect(err).Should(BeNil())

log.Info(
"Setting up relayer config",
"hostA", hostA,
"portA", portA,
"blockChainA", subnetAInfo.BlockchainID.String(),
"hostB", hostB,
"portB", portB,
"blockChainB", subnetBInfo.BlockchainID.String(),
"subnetA", subnetAInfo.SubnetID.String(),
"subnetB", subnetBInfo.SubnetID.String(),
)

relayerConfig := config.Config{
LogLevel: logging.Info.LowerString(),
NetworkID: peers.LocalNetworkID,
PChainAPIURL: subnetAInfo.ChainNodeURIs[0],
EncryptConnection: false,
StorageLocation: storageLocation,
SourceSubnets: []config.SourceSubnet{
{
SubnetID: subnetAInfo.SubnetID.String(),
ChainID: subnetAInfo.BlockchainID.String(),
VM: config.EVM.String(),
EncryptConnection: false,
APINodeHost: hostA,
APINodePort: portA,
MessageContracts: map[string]config.MessageProtocolConfig{
teleporterContractAddress.Hex(): {
MessageFormat: config.TELEPORTER.String(),
Settings: map[string]interface{}{
"reward-address": fundedAddress.Hex(),
},
},
},
},
},
DestinationSubnets: []config.DestinationSubnet{
{
SubnetID: subnetBInfo.SubnetID.String(),
ChainID: subnetBInfo.BlockchainID.String(),
VM: config.EVM.String(),
EncryptConnection: false,
APINodeHost: hostB,
APINodePort: portB,
AccountPrivateKey: hex.EncodeToString(fundedKey.D.Bytes()),
},
},
}

data, err := json.MarshalIndent(relayerConfig, "", "\t")
Expect(err).Should(BeNil())

f, err := os.CreateTemp(os.TempDir(), "relayer-config.json")
Expect(err).Should(BeNil())

_, err = f.Write(data)
Expect(err).Should(BeNil())
relayerConfigPath := f.Name()

log.Info("Created awm-relayer config", "configPath", relayerConfigPath, "config", string(data))

//
// Build Relayer
//
// Build the awm-relayer binary
cmd := exec.Command("./scripts/build.sh")
out, err := cmd.CombinedOutput()
fmt.Println(string(out))
Expect(err).Should(BeNil())

//
// Send a transaction to Subnet A to issue a Warp Message from the Teleporter contract to Subnet B
//
log.Info("Sending transaction from Subnet A to Subnet B")
ctx := context.Background()

relayerCmd, relayerCancel = testUtils.RunRelayerExecutable(ctx, relayerConfigPath)

log.Info("Packing teleporter message")
payload, err = teleporter.PackSendCrossChainMessageEvent(common.Hash(subnetBInfo.BlockchainID), teleporterMessage)
Expect(err).Should(BeNil())

input := teleporterTestUtils.SendCrossChainMessageInput{
DestinationChainID: subnetBInfo.BlockchainID,
DestinationAddress: teleporterMessage.DestinationAddress,
FeeInfo: teleporterTestUtils.FeeInfo{
ContractAddress: fundedAddress,
Amount: big.NewInt(0),
},
RequiredGasLimit: teleporterMessage.RequiredGasLimit,
AllowedRelayerAddresses: teleporterMessage.AllowedRelayerAddresses,
Message: teleporterMessage.Message,
}

// Send a transaction to the Teleporter contract
signedTx := teleporterTestUtils.CreateSendCrossChainMessageTransaction(ctx, subnetAInfo, input, fundedAddress, fundedKey, teleporterContractAddress)

// Sleep for some time to make sure relayer has started up and subscribed.
time.Sleep(15 * time.Second)
log.Info("Subscribing to new heads on destination chain")

newHeadsB := make(chan *types.Header, 10)
sub, err := subnetBInfo.ChainWSClient.SubscribeNewHead(ctx, newHeadsB)
Expect(err).Should(BeNil())
defer sub.Unsubscribe()

log.Info("Sending teleporter transaction", "destinationChainID", subnetBInfo.BlockchainID, "txHash", signedTx.Hash())
receipt := teleporterTestUtils.SendTransactionAndWaitForAcceptance(ctx, subnetAInfo.ChainWSClient, signedTx)

bind, err := teleportermessenger.NewTeleportermessenger(teleporterContractAddress, subnetAInfo.ChainWSClient)
Expect(err).Should(BeNil())
sendEvent, err := teleporterTestUtils.GetSendEventFromLogs(receipt.Logs, bind)
Expect(err).Should(BeNil())
Expect(sendEvent.DestinationChainID[:]).Should(Equal(subnetBInfo.BlockchainID[:]))

teleporterMessageID := sendEvent.Message.MessageID

// Get the latest block from Subnet B
log.Info("Waiting for new block confirmation")
newHead := <-newHeadsB
log.Info("Received new head", "height", newHead.Number.Uint64())
blockHash := newHead.Hash()
block, err := subnetBInfo.ChainWSClient.BlockByHash(ctx, blockHash)
Expect(err).Should(BeNil())
log.Info(
"Got block",
"blockHash", blockHash,
"blockNumber", block.NumberU64(),
"transactions", block.Transactions(),
"numTransactions", len(block.Transactions()),
"block", block,
)
accessLists := block.Transactions()[0].AccessList()
Expect(len(accessLists)).Should(Equal(1))
Expect(accessLists[0].Address).Should(Equal(warp.Module.Address))

// Check the transaction storage key has warp message we're expecting
storageKeyHashes := accessLists[0].StorageKeys
packedPredicate := predicateutils.HashSliceToBytes(storageKeyHashes)
predicateBytes, err := predicateutils.UnpackPredicate(packedPredicate)
Expect(err).Should(BeNil())
receivedWarpMessage, err = avalancheWarp.ParseMessage(predicateBytes)
Expect(err).Should(BeNil())

// Check that the transaction has successful receipt status
txHash := block.Transactions()[0].Hash()
receipt, err = subnetBInfo.ChainWSClient.TransactionReceipt(ctx, txHash)
Expect(err).Should(BeNil())
Expect(receipt.Status).Should(Equal(types.ReceiptStatusSuccessful))

// Check that the transaction emits ReceiveCrossChainMessage
bind, err = teleportermessenger.NewTeleportermessenger(teleporterContractAddress, subnetBInfo.ChainWSClient)
Expect(err).Should(BeNil())

receiveEvent, err := teleporterTestUtils.GetReceiveEventFromLogs(receipt.Logs, bind)
Expect(err).Should(BeNil())
Expect(receiveEvent.OriginChainID[:]).Should(Equal(subnetAInfo.BlockchainID[:]))
Expect(receiveEvent.Message.MessageID.Uint64()).Should(Equal(teleporterMessageID.Uint64()))

log.Info("Finished sending warp message, closing down output channel")

// Cancel the command and stop the relayer
relayerCancel()
_ = relayerCmd.Wait()

//
// Validate Received Warp Message Values
//
log.Info("Validating received warp message")
Expect(receivedWarpMessage.SourceChainID).Should(Equal(subnetAInfo.BlockchainID))
addressedPayload, err := warpPayload.ParseAddressedPayload(receivedWarpMessage.Payload)
Expect(err).Should(BeNil())

receivedDestinationID, err := ids.ToID(addressedPayload.DestinationChainID.Bytes())
Expect(err).Should(BeNil())
Expect(receivedDestinationID).Should(Equal(subnetBInfo.BlockchainID))
Expect(addressedPayload.DestinationAddress).Should(Equal(teleporterContractAddress))
Expect(addressedPayload.Payload).Should(Equal(payload))

// Check that the teleporter message is correct
receivedTeleporterMessage, err := teleporter.UnpackTeleporterMessage(addressedPayload.Payload)
Expect(err).Should(BeNil())
Expect(*receivedTeleporterMessage).Should(Equal(teleporterMessage))

//
// Try Relaying Already Delivered Message
//
log.Info("Creating new relayer instance to test already delivered message")
logger := logging.NewLogger(
"awm-relayer",
logging.NewWrappedCore(
logging.Info,
os.Stdout,
logging.JSON.ConsoleEncoder(),
),
)
jsonDB, err := database.NewJSONFileStorage(logger, storageLocation, []ids.ID{subnetAInfo.BlockchainID, subnetBInfo.BlockchainID})
Expect(err).Should(BeNil())

// Modify the JSON database to force the relayer to re-process old blocks
jsonDB.Put(subnetAInfo.BlockchainID, []byte(database.LatestProcessedBlockKey), []byte("0"))
jsonDB.Put(subnetBInfo.BlockchainID, []byte(database.LatestProcessedBlockKey), []byte("0"))

// Run the relayer
relayerCmd, relayerCancel = testUtils.RunRelayerExecutable(ctx, relayerConfigPath)

// We should not receive a new block on subnet B, since the relayer should have seen the Teleporter message was already delivered
log.Info("Waiting for 10s to ensure no new block confirmations on destination chain")
Consistently(newHeadsB, 10*time.Second, 500*time.Millisecond).ShouldNot(Receive())

// Cancel the command and stop the relayer
relayerCancel()
_ = relayerCmd.Wait()
}
Loading

0 comments on commit 8908f1d

Please sign in to comment.