Skip to content

Commit

Permalink
Add OpenEthereum eth1 node option
Browse files Browse the repository at this point in the history
  • Loading branch information
yorickdowne committed Nov 6, 2020
1 parent 3c359fd commit f3db25d
Show file tree
Hide file tree
Showing 7 changed files with 162 additions and 47 deletions.
25 changes: 14 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ Currently supported clients:
- Nimbus

Currently supported optional components:
- geth, local eth1 node. Use this or a 3rd-party provider of eth1 chain data to "feed"
your eth2 beacon node, so you can [propose](https://ethos.dev/beacon-chain/) blocks.
- openethereum, local eth1 node
- geth, local eth1 node.
> Use one of the local eth1 node options or a 3rd-party provider of eth1 chain data to "feed"
> your eth2 beacon node, so you can [propose](https://ethos.dev/beacon-chain/) blocks.
- slasher, Running slasher is optional, but helps secure the chain and may result in additional earnings.
- Grafana dashboard

Expand Down Expand Up @@ -128,11 +130,11 @@ sudo docker-compose up -d eth2
> **Nimbus and Teku**: Beacon and validator run in the same process, there is only one container for both
If, however, you chose not to store the wallet password with the validator, you will need
to bring the beacon and, if in use, geth, up individually instead, then "run"
to bring the beacon and, if in use, eth1, up individually instead, then "run"
the validator so it can prompt you for input:

```
sudo docker-compose up -d geth beacon
sudo docker-compose up -d eth1 beacon
sudo docker-compose run validator
```

Expand Down Expand Up @@ -186,7 +188,7 @@ service.
## Addendum: Monitor the client

Monitoring the logs of the client is useful for troubleshooting
and to judge the amount of time left before the beacon and geth nodes
and to judge the amount of time left before the beacon and eth1 nodes
are fully synchronized.

To see a list of running containers:
Expand Down Expand Up @@ -223,14 +225,14 @@ Inside the project directory, run:<br />
Then `cp .env .env.bak` and `cp default.env .env`, and set variables inside `.env`
the way you need them, with `.env.bak` as a guide.

### Geth
### Eth1

Run:<br />
`sudo docker-compose build --no-cache geth`
`sudo docker-compose build --no-cache eth1`

Then stop, remove and start geth:<br />
`sudo docker-compose stop geth && sudo docker-compose rm geth`<br />
`sudo docker-compose up -d geth`
Then stop, remove and start eth1:<br />
`sudo docker-compose stop eth1 && sudo docker-compose rm eth1`<br />
`sudo docker-compose up -d eth1`

### Client

Expand Down Expand Up @@ -329,7 +331,8 @@ key sequence to detach from it again.
`sudo docker ps` lists all running services, with the container name to the right.<br />
`sudo docker logs containername` shows logs for a container, `sudo docker logs -f containername` scrolls them.<br />
`sudo docker-compose logs servicename` shows logs for a service, `sudo docker-compose logs -f servicename` scrolls them.<br />
`sudo docker exec -it containername /bin/bash` will connect you to a running service in a bash shell. The geth service doesn't have a shell.<br />
`sudo docker exec -it containername /bin/bash` will connect you to a running service in a bash shell. The eth1 service doesn't have a shell
if using geth.<br />

You may start a service with `sudo docker-compose up -d servicename` and then find it's not in `sudo docker ps`. That means it terminated while
trying to start. To investigate, you could leave the `-d` off so you see logs on command line:<br />
Expand Down
27 changes: 15 additions & 12 deletions RECOMMENDATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ You'd get yourself slashed, and no-one wants that. Protecting you from this
is a work in progress. Choose one client, and one client only, and run that.

**You need an eth1 source**<br />
This project assumes you'll use geth. It doesn't have to be that, it can
This project assumes you'll use openethereum or geth. It doesn't have to be that, it can
be a 3rd party. You need some source for eth1, so that your validator can
successfully propose blocks.

Expand All @@ -23,18 +23,19 @@ you will need to open in `ufw` depend on the client you choose.

## Firewalling

geth: 30303 tcp/udp, forwarded to your server<br />
eth1: 30303 tcp/udp, forwarded to your server<br />
lighthouse: 9000 tcp/udp, forwarded to your server<br />
prysm: 13000 tcp and 12000 udp, forwarded to your server<br />
grafana: 3000 tcp, open on ufw but not forwarded to your server.<br />
The grafana port is insecure http:// and should only be accessed locally.
For cloud-hosted instances, a reverse proxy such as nginx or
traefik can be used.
> The grafana port is insecure http:// and should only be accessed locally.
> For cloud-hosted instances, a reverse proxy such as nginx or
> traefik can be used. An [SSH tunnel](https://www.howtogeek.com/168145/how-to-use-ssh-tunneling/)
> is also a great option.
## Before depositing

You likely want to wait to deposit your eth until you can see in the logs
that the eth1 node (e.g. geth) is synchronized and the eth2 beacon node
that the eth1 node (e.g. openethereum) is synchronized and the eth2 beacon node
is fully synchronized, which happens after that. This takes hours on
testnet and could take days on mainnet.

Expand Down Expand Up @@ -98,12 +99,14 @@ For more on withdrawal key security, read this article: https://www.attestant.io

## Resources, hardware

See the client team recommendations. Generally, however, 8 GiB of RAM is considered
a minimum, and 16 GiB is recommended. 2 or 4 CPU cores, and an SSD for storage
because the node databases are so IOPS-heavy. The eth1 node (geth) would require
around 310GiB of storage by itself. The beacon node database is small, around 11GiB,
but we don't know what growth will look like from phase 1.5 and 2 on. If
you are running a slasher, that might be another 100 to 300GiB by itself.
See the client team recommendations. Generally, however, 8 GiB of RAM is a tight
fit, and 16 GiB is recommended. Some clients such as Teku may need more RAM out
of the box. 2 or 4 CPU cores, and an SSD for storage because the node databases
are so IOPS-heavy. The Geth eth1 node would require around 330GiB of storage by
itself initially, which can grow to 500 GiB over a year. The OpenEthereum eth1
node would require 110 to 200GiB of storage. The beacon node database is small,
around 11GiB, but we don't know what growth will look like from phase 1.5 and 2 on.
If you are running a slasher, that might be another 100 to 300GiB by itself.

Two home server builds that I like and am happy to recommend are below. Both support
IPMI, which means they can be managed and power-cycled remotely and need neither
Expand Down
20 changes: 11 additions & 9 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,16 @@ Please choose:
* Lighthouse
* Prysm
* Teku
* Nimbus - as of 10/09 requires an archive-mode geth or remote archive like goerli.infura.io
* Nimbus
* Your source of eth1 data
* openethereum
* geth
* 3rd-party
* Whether to run a slasher (not yet implemented)
* Whether to run a slasher (experimental for Prysm)
* Whether to run a grafana dashboard for monitoring

> Note: Teku is written in Java, which makes it memory-hungry. In its default configuration, you may
> want a machine with 16 GiB of RAM. See `.env` for a parameter to restrict Teku to 6 GiB of heap. It
> want a machine with 24 GiB of RAM or more. See `.env` for a parameter to restrict Teku to 6 GiB of heap. It
> may still take more than 6 GiB of RAM in total.
First, copy the environment file.<br />
Expand Down Expand Up @@ -89,26 +90,27 @@ exposed to the host, and for everything but Grafana to the Internet via your fir
### Client compose files

Set the `COMPOSE_FILE` string depending on which client you are going to use. Add optional services like
geth with `:` between the file names.
openethereum with `:` between the file names.
- `lh-base.yml` - Lighthouse
- `prysm-base.yml` - Prysm
- `teku-base.yml` - Teku
- `nimbus-base.yml` - Nimbus
- `oe.yml` - local openethereum eth1 chain node
- `geth.yml` - local geth eth1 chain node
- `prysm-slasher.yml` - Prysm experimental Slasher which helps secure the chain and may result in additional earnings. The experimental slasher can lead to missed attestations do to the additional resource demand.
- `lh-grafana.yml` - grafana dashboard for Lighthouse
- `prysm-grafana.yml` - grafana dashboard for Prysm. Not encrypted, do not expose to Internet.
- `prysm-web.yml` - Prysm experimental Web UI and Grafana dashboard. Not encrypted, do not expose to Internet. **Mutually exclusive** with `prysm-grafana.yml`
- `prysm-slasher.yml` - Prysm experimental Slasher which helps secure the chain and may result in additional earnings. The experimental slasher can lead to miss attestations do to the additional resource demand.
- `nimbus-grafana.yml` - grafana dashboard for Nimbus
- `teku-grafana.yml` - grafana dashboard for Teku

For example, Lighthouse with local geth and grafana:
`COMPOSE_FILE=lh-base.yml:geth.yml:lh-grafana.yml`
For example, Lighthouse with local openethereum and grafana:
`COMPOSE_FILE=lh-base.yml:oe.yml:lh-grafana.yml`

> See [WEB](WEB.md) for notes on using the experimental Prysm Web UI
In this setup, clients are isolated from each other. Each run their own validator, and if geth
is in use, their own geth. This is perfect for running a single client, or multiple isolated
In this setup, clients are isolated from each other. Each run their own validator, and if eth1
is in use, their own eth1 node. This is perfect for running a single client, or multiple isolated
clients each in their own directory.

If you want to run multiple isolated clients, just clone this project into a new directory for
Expand Down
20 changes: 14 additions & 6 deletions default.env
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
# This is vital for key import to work. You can find your UID with "echo $UID".
LOCAL_UID=1000
# Client choice: See SETUP.md for available options
COMPOSE_FILE=lh-base.yml:geth.yml:lh-grafana.yml
# ETH1 endpoint / chain source. This default uses the geth container
ETH1_NODE=http://geth:8545
COMPOSE_FILE=lh-base.yml:oe.yml:lh-grafana.yml
# ETH1 endpoint / chain source. This default uses the eth1 node container
ETH1_NODE=http://eth1:8545
# Graffiti to use for validator, in order to receive POAP
GRAFFITI=eth2-docker
# Number of validators to create with deposit-cli
Expand All @@ -22,12 +22,15 @@ LOG_LEVEL=info
NIM_METRICS=NIMFLAGS="-d:insecure"
# Network to use for eth2. Set to mainnet for main net.
NETWORK=medalla
# Network to use for geth. Comment out for main net.
ETH_NETWORK=--goerli
# Network to use for eth1. If using OpenEthereum, set to "ethereum" for main net.
ETH1_NETWORK=goerli
# If using Geth as the eth1 node, comment out for main net.
GETH1_NETWORK=--goerli


# Ports you will forward to your staking node. Adjust here if you are
# going to use something other than defaults.
GETH_PORT=30303
ETH1_PORT=30303
LH_PORT=9000
PRYSM_PORT=13000
PRYSM_UDP_PORT=12000
Expand All @@ -41,6 +44,11 @@ GRAFANA_PORT=3000
GETH_BUILD_TARGET=$(git describe --tags $(git rev-list --tags --max-count=1))
GETH_USER=geth

# Build target dev until #74 has been pulled into main. At that point, switch to release tracking
OE_BUILD_TARGET=dev
#OE_BUILD_TARGET=$(git describe --tags $(git rev-list --tags --max-count=1))
OE_USER=openethereum

LH_BUILD_TARGET=$(git describe --tags $(git rev-list --tags --max-count=1))
LH_USER=lighthouse

Expand Down
18 changes: 9 additions & 9 deletions geth.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
version: "3"
services:
geth:
eth1:
restart: "${RESTART}"
user: ${LOCAL_UID}:${LOCAL_UID}
build:
Expand All @@ -11,10 +11,10 @@ services:
- UID=${LOCAL_UID}
image: geth
volumes:
- geth-data:/var/lib/goethereum
- eth1-data:/var/lib/goethereum
ports:
- ${GETH_PORT}:${GETH_PORT}/tcp
- ${GETH_PORT}:${GETH_PORT}/udp
- ${ETH1_PORT}:${ETH1_PORT}/tcp
- ${ETH1_PORT}:${ETH1_PORT}/udp
expose:
- 8545
- 8546
Expand All @@ -31,13 +31,13 @@ services:
- --datadir
- /var/lib/goethereum
- --port
- ${GETH_PORT}
command: ${ETH_NETWORK}
- ${ETH1_PORT}
command: ${GETH1_NETWORK}
beacon:
depends_on:
- geth
- eth1
eth2:
depends_on:
- geth
- eth1
volumes:
geth-data:
eth1-data:
39 changes: 39 additions & 0 deletions oe.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
version: "3"
services:
eth1:
restart: "${RESTART}"
user: ${LOCAL_UID}:${LOCAL_UID}
build:
context: ./openethereum
args:
- BUILD_TARGET=${OE_BUILD_TARGET}
- USER=${OE_USER}
- UID=${LOCAL_UID}
image: openethereum
volumes:
- eth1-data:/var/lib/openethereum
ports:
- ${ETH1_PORT}:${ETH1_PORT}/tcp
- ${ETH1_PORT}:${ETH1_PORT}/udp
expose:
- 8545
entrypoint:
- openethereum
- --no-ws
- --no-ipc
- --no-secretstore
- --base-path=/var/lib/openethereum
- --jsonrpc-interface=all
- --jsonrpc-apis=net,eth
- --jsonrpc-hosts=all
- --chain=${ETH1_NETWORK}
- --port
- ${ETH1_PORT}
beacon:
depends_on:
- eth1
eth2:
depends_on:
- eth1
volumes:
eth1-data:
60 changes: 60 additions & 0 deletions openethereum/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
FROM alpine:edge AS builder

# show backtraces
ENV RUST_BACKTRACE 1

ARG BUILD_TARGET

RUN apk update && apk add --no-cache \
build-base \
cargo \
cmake \
eudev-dev \
linux-headers \
perl \
rust \
git \
bash

WORKDIR /
RUN bash -c "git clone https://github.com/openethereum/openethereum.git && cd openethereum && git config advice.detachedHead false && git fetch --all --tags && git checkout ${BUILD_TARGET} && cargo build --release --features final --target x86_64-alpine-linux-musl --verbose && strip target/x86_64-alpine-linux-musl/release/openethereum"

FROM alpine:edge

# show backtraces
ENV RUST_BACKTRACE 1

ARG USER
ARG UID

# curl and jq are installed to help create health and readiness checks on Kubernetes
RUN apk update && apk add --no-cache \
libstdc++ \
eudev-libs \
libgcc \
curl \
jq

# See https://stackoverflow.com/a/55757473/12429735RUN
RUN adduser \
--disabled-password \
--gecos "" \
--home "/nonexistent" \
--shell "/sbin/nologin" \
--no-create-home \
--uid "${UID}" \
"${USER}"

# Create data mount point with permissions
RUN mkdir -p /var/lib/openethereum && chown ${USER}:${USER} /var/lib/openethereum && chmod 700 /var/lib/openethereum

# Use an unprivileged user.
USER ${USER}:${USER}

EXPOSE 8080 8545 8180

WORKDIR /var/lib/openethereum

COPY --chown=openethereum:openethereum --from=builder /openethereum/target/x86_64-alpine-linux-musl/release/openethereum /usr/local/bin

ENTRYPOINT ["openethereum", "--base-path=/var/lib/openethereum"]

0 comments on commit f3db25d

Please sign in to comment.