Skip to content

Latest commit

 

History

History
208 lines (154 loc) · 9.26 KB

creating-premerge-new-testnet.md

File metadata and controls

208 lines (154 loc) · 9.26 KB
Raw

Creating a new multi-client, multi-party testnet

This runbook explains the steps involved in running a multi-client testnet involving external parties. The basic steps are:

  • Decide date for genesis + delay + fork epoch
  • Decide on number of validators
  • Contract
  • Setup config
  • Generate mnemonics
  • Generate genesis state
  • Send mnemonics
  • Deploy 1st client or bootnode
  • Get ENR
  • PR with the information
  • Provision nodes
  • Deploy other clients with ENR to get them peering
  • Check if genesis root + genesis data matches with client API. It should match what the python script (genesis testnet tool repo) outputs for our genesis.ssz
  • Publish list of ENR address of the deployed clients, add as file to the PR
  • Setup monitoring and dashboards for the clients
  • Setup forkmon
  • Setup beacon explorer using a prysm node

Decide config data

Talk to the other clients/participants as to a comfortable time for everyone. 11/12 UTC is a good timezone to attempt genesis, everyone would be awake at that timezone. Plan in at least a few days of delay to allow client teams to bake in genesis and test in their own internal systems + timezone lag.

Things that need to be decided from an altair hf perspective: min genesis, genesis delay, fork epoch

general formula:

- --timestamp = when nodes can start peering (in eth2-testnet-genesis tool, for generating genesis state)
- GENESIS_DELAY = added to timestamp option, to get the resulting genesis_time in the state.
- genesis_time = The actual moment the network is considered live and epoch 0 starts
- genesis_time + ALTAIR_FORK_EPOCH * SLOTS_PER_EPOCH = time of the altair hardfork

Decide on number of validators

Depending on the importance of the testnet, decide the number of participating validators. An internal testnet could have just 1000 vals. A multi-client multi-party should ideally start at 10,000 vals to allow for an accurate division.

Deposit contract

We will re-use the deposit contract and its associated tools found in this repository: https://github.com/ethereum/eth2.0-specs

  • Clone the repo
  • Run make compile_deposit_contract, this should generate a build folder with the
  • Get your metamask/eth1 address and private key, export it as variables called ETH1_FROM_ADDR and ETH1_FROM_PRIV(google how to get private key from metamask)
  • Move into the build folder with cd build
  • Deploy the contract with ethereal contract deploy --network=goerli --name=DepositContract --json=combined.json --from=$ETH1_FROM_ADDR --privatekey=$ETH1_FROM_PRIV

You should now see a tx hash, enter that in a block explorer for goerli and you should see the deployed contract.

At this stage, the following are important to note for the future:

  • Tx hash of the deployment
  • Contract address
  • Deposit block number
  • Deposit block hash

Setup config

We are now ready to generate the testnet config. Find the latest version of the config here: https://github.com/eth2-clients/eth2-networks.

  • Create a branch in the repo, create a new folder in the appropriate location
  • Copy over the base config and make edits to the following fields:
    • CONFIG_NAME
    • MIN_GENESIS_ACTIVE_VALIDATOR_COUNT
    • MIN_GENESIS_TIME
    • GENESIS_FORK_VERSION
    • GENESIS_DELAY
    • ALTAIR_FORK_EPOCH
    • DEPOSIT_CONTRACT_ADDRESS

Generate mnemonics

Now that the deposit contract and config are set, we know how many validators we need and where to make the deposits.

Typically, for a testnet, all the mnemonics are generated by 1 party and then sent to the teams running the validators.

  • Mnemonics can be generated by using the eth2-val-tools cli tool
  • Create a mnemonics.yaml file (DO NOT COMMIT!)
  • Enter the mnemonics as a list:
# Who it's being assigned to
- mnemonic: ""  # a 24 word BIP 39 mnemonic
  count: 1234  # amount of validators

# Who it's being assigned to
- mnemonic: ""  # a 24 word BIP 39 mnemonic
  count: 1234  # amount of validators
  • The count numbers should add up to the MIN_GENESIS_ACTIVE_VALIDATOR_COUNT field in the config

Generate genesis state

We now have all the component parts to generate the genesis state. Since we want to avoid making deposits for all the validators individually, we can just bake the deposits into the genesis state.

This can be done with the eth2-testnet-genesis tool and it requires the following parameters:
- phase0/altair/merge: Specify which genesis state to create, for altair hf we use phase0 since we want to simulate the fork
- config: The config file created earlier that specifies the chain spec
- eth1-block: The deposit block hash, block hash of the block in which the contract was deposited
- mnemonics: The mnemonic file created earlier, specifying the mnemonic and associated validator count
- timestamp: When the clients can start peering (NOT GENESIS TIME). The timestamp value + genesis delay = genesis time

You should now see a genesis.ssz file that contains the genesis state.

Confirm the genesis state data is correct with: zcli pretty phase0 BeaconState genesis.ssz

Send mnemonics

Reach out to the client teams and send their mnemonic + validator counts.

Deploy 1st client or bootnode

Now that we have a setup network, we need to setup the bootnode so peers can find each other.

To do this:

  • Create a AWS instance to host the bootnode, ideally in the cluster repo
  • Create an inventory, ideally in eth2.0-devops/testnets/<testnet-name>-bootnodes
  • Export the private key for the bootnode with export <testnetname>_BOOTNODE_PRIV_101="xxxxxxx"
  • Add the variables needed for the bootnode, update the GENESIS_FORK_VERSION to reflect the new testnet
  • Run the setup_network_bootnodes.yml playbook in ansible_eth2/playbooks to setup the bootnode with the config
  • SSH into the bootnode and check the logs with docker logs bootnode
  • Grab the field called ENR
  • Confirm the fields using https://enr-viewer.com/

PR with the information

We should have all the information required by the client teams for their individual setups. So we create a PR in the eth2-clients/eth2-networks repo.

The following files need to be added in the respective folder for the testnet:

  • README.md: One liner for the testnet
  • bootstrap_nodes.txt: Containing the ENR of the bootnodes or ready clients
  • config.yaml: The config for the testnet
  • deploy_block.txt: The block at which the deposit contract was deployed
  • deposit_contract.txt: The deposit contract address
  • genesis.ssz: The genesis file created previously

Provision nodes

The planning side of the testnet is now done. Provision the nodes as usual, Terraform on cluster repo + Ansible in eth2.0-devops.

Deploy other clients with ENR to get them peering

Use the config to create keys + deploy clients in the wished % division.

Check if genesis data is correct on clients

  • SSH into the AWS instances and check the client genesis data with an API request to the beacon node. This should work: curl localhost:4000/eth/v1/beacon/genesis
  • Use the compute_genesis_details.py file in the `testnet_config folder to cross-check the data

Publish list of ENR address

Update the bootstrap_nodes.txt file in the PR with some more client ENRs, if they are stable.

Setup monitoring and dashboards for the clients

Set the monitoring variables (push_metrics_enabled: true) and check the data in grafana.

Copy the existing dashboards and change the datasource to match the new filter.

Setup forkmon

Forkmon allows us to check latest slot and forks in realtime.

  • Ensure terraform allows access to the beacon API port of the client
  • Use iptables IP filtering to just allow access to the port from the forkmon instance
  • SSH into 54.208.237.210
  • Copy the config config.altair-dev.yaml
  • Edit the information with the new IP addresses
  • Update Caddyfile with the new DNS
  • Run docker run -d --restart always --network some-net --name eth2-fork-mon-<TESTNETNAME> -v /home/ubuntu/config.<TESTNETNAME>.yaml:/config.yaml ralexstokes/eth2-fork-mon
  • Restart caddy with docker run -d --restart always --network some-net --name caddy -p 80:80 -p 443:443 -v $(pwd)/Caddyfile:/etc/Caddyfile --link eth2-fork-mon:eth2-fork-mon --link eth2-fork-mon-altair-dev:eth2-fork-mon-altair-dev abiosoft/caddy

Look into replacing this with an ansible playbook

If forkmon doesn't pick up a node (lighthouse pre-genesis, its normal, just restart it post genesis).

Setup beacon explorer

  • You need to have a prysm node for this
  • Instructions to come ...
https://github.com/gobitfly/eth2-beaconchain-explorer
https://github.com/protolambda/mergenet-ansible
- a prysm GRPC endpoint
- configure prysm to take state snapshots on short interval (there's some archive related cli flag for that)
- install postgress with ansible
- build a docker image of the explorer, without the merge features (I can hack in altair features later)
- configure the explorer (I don't have access to original mergenet config, but will help figure that out)
- use the ansible to deploy (same machine as prysm node)

  setup_machine.yml
  upload_explorer_data.yml
  start_explorer_net.yml
  start_explorer_postgres.yml
  init_explorer_sql_tables.yml
  start_explorer.yml
  start_explorer_statistics.yml