Skip to content

Latest commit

 

History

History
560 lines (423 loc) · 18.7 KB

run-random-beacon.adoc

File metadata and controls

560 lines (423 loc) · 18.7 KB

Run Random Beacon

1. System Considerations

The Keep Network expects certain capabilites for each node running on the network. To help attain these capabilities consider the following criteria:

  • It is paramount that Keep nodes remain available to the Keep Network. We strongly encourage a stable and redundant internet connection.

  • A connection to a production grade self-hosted or third party Ethereum node deployment.

  • Persistent and redundant storage that will survive a VM or container rotation, and disk failure.

  • Each random beacon client running on the network requires a unique Ethereum operator account.

  • Each random beacon client running on the network requires a unique IP address or a unique application port running under the same IP.

  • Recommended machine types by provider:

Your operating environment will ultimately dictate what machine type to go with. This is particulary relevant if you’re running a containerized solution where multiple applications are sharing VM resources. The below types are sufficient for running at least one instance of the Keep Random Beacon client.

Cloud Provider Machine Type

Google Cloud

n1-standard-2

AWS

t3.medium

Azure

A2 v2

Self-hosted

2 vCPU / 4 GiB RAM / 1 GiB Persistent Storage

2. Gas Costs

Random Beacon smart contracts reimburse the operator for successfully submitting relay entry and DKG result but they do not reimburse for submitting a group selection ticket. Reimbursements are sent to the beneficiary account and can be claimed along with rewards once the group expires. It is expected that the operators have enough ETH on the accounts used by clients to submit the required transactions and that the operator account balance is monitored and refilled as needed. Bear in mind that the higher stake is, the operator is selected more frequently and is expected to submit more transactions as a result.

Below is the average gas cost of the most important transactions the client is submitting:

TX Gas Cost Reimbursed

Submit group selection ticket

140 000

No

Submit DKG result

1 740 000

Yes

Submit relay entry

280 000

Yes

For example, if the operator has 10 x minimum stake, it can submit 10 tickets. If the operator has been selected to the group with index 1, it is expected to submit DKG result and every relay entry the group will produce. Assuming the group produces 100 entries, the cost for the operator is (10 * 140 000 + 1 740 000 + 100 * 280 000) * gas_price ETH. It means that the operator needs to have on their account:

  • For the gas price of 20 Gwei, at least 0.6228 ETH.

  • For the gas price of 100 Gwei, at least 3.114 ETH.

  • For the gas price of 800 Gwei, at least 24.912 ETH.

It is paramount that the operator accounts have some safety margin and consider the current gas price and stake when funding their accounts.

3. Configuration

3.1. Network

Default port mappings.

Egress Port

Ethereum Network

8545 / 8546

Keep Network

3919

Ingress Port

Keep Network

3919

If you set a different port in your keep-client configuration, or configure peers with non-default ports configured, firewall rules will need to be adjusted accordingly.

3.2. Application

Application configurations are stored in a .toml file and passed to the application run command with the --config flag.

3.2.1. Sample

# Ethereum host connection info.
[ethereum]
  URL = "ws://127.0.0.1:8546"
  URLRPC = "http://127.0.0.1:8545"

# Keep operator Ethereum account.
[ethereum.account]
  Address = "0xAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8AAAAAAAAA"
  KeyFile = "/Users/someuser/ethereum/data/keystore/UTC--2018-03-11T01-37-33.202765887Z--AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8AAAAAAAAA"

# Keep contract addresses configuration.
[ethereum.ContractAddresses]
  # Hex-encoded address of KeepRandomBeaconOperator contract
  KeepRandomBeaconOperator = "0xBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
  # Hex-encoded address of TokenStaking contract
  TokenStaking = "0xCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC"
  # Hex-encoded address of KeepRandomBeaconService contract. Only needed
  # in cases where the client's utility functions will be used (e.g., the
  # relay subcommand).
  KeepRandomBeaconService = "0xDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD"

# Keep network configuration.
[LibP2P]
  Peers = ["/ip4/127.0.0.1/tcp/3919/ipfs/njOXcNpVTweO3fmX72OTgDX9lfb1AYiiq4BN6Da1tFy9nT3sRT2h1", "/dns4/some-keep-host.com/tcp/3919/ipfs/njOXcNpVTweO3fmX72OTgDX9lfb1AYiiq4BN6Da1tFy9nT3sRT2h1"]
  Port  = 3919
  # Override the node's default addresses announced in the network
  AnnouncedAddresses = ["/dns4/example.com/tcp/3919", "/ip4/80.70.60.50/tcp/3919"]

# Storage is encrypted
[Storage]
  DataDir = "/my/secure/location"

3.2.2. Parameters

ethereum Description Default Required

URL

The Ethereum host your keep-client will connect to. Websocket protocol/port.

""

Yes

URLRPC

The Ethereum host your keep-client will connect to. RPC protocol/port.

""

Yes

ethereum.account Description Default Required

Address

The Keep operator Ethereum account address.

""

Yes

KeyFile

The local filesystem path to your Keep operator Ethereum account keyfile.

""

Yes

ethereum.ContractAddresses Description Default Required

KeepRandomBeaconOperator

Hex-encoded address of the KeepRandomBeaconOperator Contract.

""

Yes

KeepRandomBeaconService

Hex-encoded address of the KeepRandomBeaconService Contract.

""

Yes

TokenStaking

Hex-encoded address of the TokenStaking Contract.

""

Yes

LibP2P Description Default Required

Peers

Comma separated list of network peers to boostrap against.

[""]

Yes

Port

The port to run your instance of Keep on.

3919

Yes

AnnouncedAddresses

Multiaddr formatted hostnames or addresses annouced to the Keep Network. More on multiaddr format in the libp2p reference.

[""]

No

Storage Description Default Required

DataDir

Location to store the Keep nodes group membership details.

""

Yes

4. Build from Source

See the building section in our developer docs.

5. Docker

5.1. Get Image

Latest: docker pull keepnetwork/keep-client

Tag: docker pull keepnetwork/keep-client:<tag-version>

5.2. Run Image

This is a sample run command for illustration purposes only.

export KEEP_CLIENT_ETHEREUM_PASSWORD=$(cat .secrets/eth-account-password.txt)
export KEEP_CLIENT_CONFIG_DIR=$(pwd)/config
export KEEP_CLIENT_PERSISTENCE_DIR=$(pwd)/persistence

docker run -d \
--entrypoint /usr/local/bin/keep-client
--volume $KEEP_CLIENT_PERSISTENCE_DIR:/mnt/keep-client/persistence \
--volume $KEEP_CLIENT_CONFIG_DIR:/mnt/keep-client/config \
--env KEEP_ETHEREUM_PASSWORD=$KEEP_CLIENT_ETHEREUM_PASSWORD \
--env LOG_LEVEL=debug \
--log-opt max-size=100m \
--log-opt max-file=3 \
-p 3919:3919 \
keepnetwork/keep-client:<version> --config /mnt/keep-client/config/keep-client-config.toml start

6. Deployment Considerations

6.1. Kubernetes

At Keep we run on GCP + Kube. To accommodate the aforementioned system considerations we use the following pattern for each of our environments:

  • Regional Kube cluster.

  • 5 beacon clients, each running minimum stake required by the network.

  • A LoadBalancer Service for each client.

  • A StatefulSet for each client.

You can see our Ropsten Kube configurations here

7. Logging

Below are some of the key things to look out for to make sure you’re booted and connected to the network:

7.1. Configurable Values

LOG_LEVEL=DEBUG
IPFS_LOGGING_FMT=nocolor
GOLOG_FILE=/var/log/keep/keep.log
GOLOG_TRACING_FILE=/var/log/keep/trace.json

7.2. Startup

▓▓▌ ▓▓ ▐▓▓ ▓▓▓▓▓▓▓▓▓▓▌▐▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▄
▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▌▐▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
  ▓▓▓▓▓▓    ▓▓▓▓▓▓▓▀    ▐▓▓▓▓▓▓    ▐▓▓▓▓▓   ▓▓▓▓▓▓     ▓▓▓▓▓   ▐▓▓▓▓▓▌   ▐▓▓▓▓▓▓
  ▓▓▓▓▓▓▄▄▓▓▓▓▓▓▓▀      ▐▓▓▓▓▓▓▄▄▄▄         ▓▓▓▓▓▓▄▄▄▄         ▐▓▓▓▓▓▌   ▐▓▓▓▓▓▓
  ▓▓▓▓▓▓▓▓▓▓▓▓▓▀        ▐▓▓▓▓▓▓▓▓▓▓         ▓▓▓▓▓▓▓▓▓▓▌        ▐▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
  ▓▓▓▓▓▓▀▀▓▓▓▓▓▓▄       ▐▓▓▓▓▓▓▀▀▀▀         ▓▓▓▓▓▓▀▀▀▀         ▐▓▓▓▓▓▓▓▓▓▓▓▓▓▓▀
  ▓▓▓▓▓▓   ▀▓▓▓▓▓▓▄     ▐▓▓▓▓▓▓     ▓▓▓▓▓   ▓▓▓▓▓▓     ▓▓▓▓▓   ▐▓▓▓▓▓▌
▓▓▓▓▓▓▓▓▓▓ █▓▓▓▓▓▓▓▓▓ ▐▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓  ▓▓▓▓▓▓▓▓▓▓
▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓ ▐▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓  ▓▓▓▓▓▓▓▓▓▓

Trust math, not hardware.

-----------------------------------------------------------------------------------------------
| Keep Random Beacon Node                                                                     |
|                                                                                             |
| Port: 3919                                                                                  |
| IPs : /ip4/127.0.0.1/tcp/3919/ipfs/16Uiu2HAmCcfVpHwfBKNFbQuhvGuFXHVLQ65gB4sJm7HyrcZuLttH    |
|       /ip4/10.102.0.112/tcp/3919/ipfs/16Uiu2HAmCcfVpHwfBKNFbQuhvGuFXHVLQ65gB4sJm7HyrcZuLttH |
-----------------------------------------------------------------------------------------------

Bonus: If you want to share your LibP2P address with others you can get it from the startup log. When sharing remember to substitute the /ipv4/ address with the public facing IP of your client if you’re running on a private machine, or replace the entire /ipv4/ segment with a DNS entry if you’re using a hostname.

7.3. Peer Connections

21:19:47.129 DEBUG keep-net-w: connected to [1] peers:[16Uiu2HAm3eJtyFKAttzJ85NLMromHuRg4yyum3CREMf6CHBBV6KY]

8. ETH Networks

8.1. Mainnet

8.1.1. Boostrap Peers

"/dns4/bst-a01.core.keep.boar.network/tcp/3001/ipfs/16Uiu2HAkzYFHsqbwt64ZztWWK1hyeLntRNqWMYFiZjaKu1PZgikN",
"/dns4/bst-b01.core.keep.boar.network/tcp/3001/ipfs/16Uiu2HAkxLttmh3G8LYzAy1V1g1b3kdukzYskjpvv5DihY4wvx7D",
"/dns4/keep-boot-validator-0.prod-us-west-2.staked.cloud/tcp/3919/ipfs/16Uiu2HAmDnq9qZJH9zJJ3TR4pX1BkYHWtR2rVww24ttxQTiKhsaJ",
"/dns4/keep-boot-validator-1.prod-us-west-2.staked.cloud/tcp/3919/ipfs/16Uiu2HAmHbbMTDDsT2f6z8zMgDtJkTUDJQSYsQYUpaJjdMjiYNEf",
"/dns4/keep-boot-validator-2.prod-us-west-2.staked.cloud/tcp/3919/ipfs/16Uiu2HAmBXoNLLMYU9EcKYH6JN5tA498sXQHFWk4heK22RfXD7wC",
"/ip4/54.39.179.73/tcp/3919/ipfs/16Uiu2HAkyYtzNoWuF3ULaA7RMfVAxvfQQ9YRvRT3TK4tXmuZtaWi",
"/ip4/54.39.186.166/tcp/3919/ipfs/16Uiu2HAkzD5n4mtTSddzqVY3wPJZmtvWjARTSpr4JbDX9n9PDJRh",
"/ip4/54.39.179.134/tcp/3919/ipfs/16Uiu2HAkuxCuWA4zXnsj9R6A3b3a1TKUjQvBpAEaJ98KGdGue67p",
"/dns4/4d00662f-e56d-404a-803a-cac01ada3e15.keep.bison.run/tcp/3919/ipfs/16Uiu2HAmV3HqJjcbKMxHnDxDx4m2iEYynyYdsvU3VwaeE6Zra2P9",
"/dns4/ec1eb390-124c-4b1b-bcf7-c21709baf2b2.keep.herd.run/tcp/3919/ipfs/16Uiu2HAmVo51PqEZLADehZEbZnrp5A7qjRWFLj9E7DfwZKVhERFt",
"/dns4/2aa9b786-7360-4c22-ae73-bd95af9c11c5.keep.bison.run/tcp/3919/ipfs/16Uiu2HAm9g3QrQzSvJ8FAhgB1PmjMNgjPd3pDaJJqsdSisGsnaFe"

8.1.2. Contracts

Contract addresses needed to boot the Random Beacon client:

Token

TokenStaking

0x1293a54e160d1cd7075487898d65266081a15458

RandomBeacon

KeepRandomBeaconService

0x50510e691c90ea098e3fdd23c311731bf394aafd

KeepRandomBeaconOperator

0xdf708431162ba247ddae362d2c919e0fbafcf9de

8.2. Testnet

Keep uses the Ethereum Ropsten Testnet.

8.2.1. Faucet

The KEEP faucet will will issue a 300k KEEP token grant for the provided Ethereum account. You can use the faucet from your web browser or via a terminal using curl.

To use the faucet you need to pass your Ethereum account to the faucet endpoint with the parameter ?account=<eth-account-address>.

Curl Example:

curl 'https://us-central1-keep-test-f3e0.cloudfunctions.net/keep-faucet-ropsten?account=0x0eC14BC7cCA82c942Cf276F6BbD0413216dDB2bE'

Browser Example:

https://us-central1-keep-test-f3e0.cloudfunctions.net/keep-faucet-ropsten?account=0x0eC14BC7cCA82c942Cf276F6BbD0413216dDB2bE

Once you’ve got your KEEP token grant you can manage it with our token dashboard.

8.2.2. Bootstrap Peers

Bootstrap peers will come and go on testnet. As long as at least one of your configured peers is up, there is no need to worry.

"/dns4/bootstrap-1.core.keep.test.boar.network/tcp/3001/ipfs/16Uiu2HAkuTUKNh6HkfvWBEkftZbqZHPHi3Kak5ZUygAxvsdQ2UgG",
"/dns4/bootstrap-3.test.keep.network/tcp/3919/ipfs/16Uiu2HAm8KJX32kr3eYUhDuzwTucSfAfspnjnXNf9veVhB12t6Vf",
"/dns4/bootstrap-2.test.keep.network/tcp/3919/ipfs/16Uiu2HAmNNuCp45z5bgB8KiTHv1vHTNAVbBgxxtTFGAndageo9Dp"

8.2.3. Contracts

Contract addresses needed to boot the Random Beacon client:

Token

TokenStaking

0x9b20291F016f5CcF8956585681C6c338082925BC

RandomBeacon

KeepRandomBeaconService

0x70DB2e8bd0835FcEF45c7584938e7FD1442fabdd

KeepRandomBeaconOperator

0xC5312D5E85263362fF6283b0e9F7E2a242a4d4D8

9. Metrics

The client exposes the following metrics:

  • connected peers count,

  • connected bootstraps count,

  • Ethereum client connectivity status (if a simple read-only CALL can be executed).

Metrics can be enabled in the configuration .toml file. It is possible to customize port at which metrics endpoint is exposed as well as the frequency with which the metrics are collected.

Exposed metrics contain the value and timestamp at which they were collected.

Example metrics endpoint call result:

$ curl localhost:9601/metrics
# TYPE connected_peers_count gauge
connected_peers_count 108 1623235129569

# TYPE connected_bootstrap_count gauge
connected_bootstrap_count 10 1623235129569

# TYPE eth_connectivity gauge
eth_connectivity 1 1623235129789

10. Diagnostics

The client exposes the following diagnostics:

  • list of connected peers along with their network id and Ethereum operator address,

  • information about the client’s network id and Ethereum operator address.

Diagnostics can be enabled in the configuration .toml file. It is possible to customize port at which diagnostics endpoint is exposed.

Example diagnostics endpoint call result:

$ curl localhost:9501/diagnostics
{
  "client_info" {
   "ethereum_address":"0xDcd4199e22d09248cA2583cBDD2759b2acD22381",
   "network_id":"16Uiu2HAkzYFHsqbwt64ZztWWK1hyeLntRNqWMYFiZjaKu1PZgikN"
  },
  "connected_peers": [
    {"ethereum_address":"0x3712C6fED51CECA83cA953f6FF3458f2339436b4","network_id":"16Uiu2HAkyYtzNoWuF3ULaA7RMfVAxvfQQ9YRvRT3TK4tXmuZtaWi"},
    {"ethereum_address":"0x4bFa10B1538E8E765E995688D8EEc39C717B6797","network_id":"16Uiu2HAm9d4MG4LNrwkFmugD2pX7frm6ZmA4vE3EFAEjk7yaoeLd"},
    {"ethereum_address":"0x650A9eD18Df873cad98C88dcaC8170531cAD2399","network_id":"16Uiu2HAkvjVWogUk2gq6VTNLQdFoSHXYpobJdZyuAYeoWD66e8BD"},
    ...
  ]
}

11. Staking

11.1. Terminology

address

Hexadecimal string consisting of 40 characters prefixed with "0x" uniquely identifying Ethereum account; derived from ECDSA public key of the party. Example address: 0xb2560a01e4b8b5cb0ac549fa39c7ae255d80e943.

owner

The address owning KEEP tokens or KEEP token grant. The owner’s participation is not required in the day-to-day operations on the stake, so cold storage can be accommodated to the maximum extent.

operator

The address of a party authorized to operate in the network on behalf of a given owner. The operator handles the everyday operations on the delegated stake without actually owning the staked tokens. An operator can not simply transfer away delegated tokens, however, it should be noted that operator’s misbehaviour may result in slashing tokens and thus the entire staked amount is indeed at stake.

beneficiary

the address where the rewards for participation and all reimbursements are sent, earned by an operator, on behalf of an owner

delegated stake

an owner’s staked tokens, delegated to the operator by the owner. Delegation enables KEEP owners to have their wallets offline and their stake operated by operators on their behalf.

operator contract

Ethereum smart contract handling operations that may have an impact on staked tokens.

authorizer

the address appointed by owner to authorize operator contract on behalf of the owner. Operator contract must be pre-approved by authorizer before the operator is eligible to use it and join the specific part of the network.

11.2. Delegating tokens

KEEP tokens are delegated by the owner. During the delegation, the owner needs to appoint an operator, beneficiary, and authorizer. Owner may delegate owned tokens or tokens from a grant. Owner may decide to delegate just a portion of owned tokens or just a part of tokens from a grant. Owner may delegate multiple times to different operators. Tokens can be delegated using Tokens page in KEEP token dashboard and a certain minimum stake defined by the system is required to be provided in the delegation. The more stake is delegated, the higher chance to be selected to relay group.

Delegation takes immediate effect but can be cancelled within 12 hours without additional delay. After 12 hours operator appointed during the delegation becomes eligible for work selection.

11.3. Authorizations

Before operator is considered as eligible for work selection, authorizer appointed during the delegation needs to review and authorize Keep Random Beacon smart contract. Smart contracts can be authorized using KEEP token dashboard. Authorized operator contracts may slash or seize tokens in case of operator’s misbehavior.