Improved documentation

PREREQUISITES.md

@@ -39,15 +39,8 @@ MacOS has not been tested, if you have the ability to, please get in touch via t
 ## Windows 10 discouraged
 
 While it is technically possible to run this project, and thus a node, on Windows 10,
-I want to discourage that idea. Windows 10 is fine as an SSH client to connect *to*
-your Linux server, but not as a basis for the node server itself.
+I want to [discourage that idea](WINDOWS.md). Windows 10 is fine as an SSH client to connect *to*
+your Linux server, but not as a basis to run the node server itself inside Docker.
 
-In testing, Windows 10 time synchronization was less than accurate, and WSL2 would lose
-time sync when a machine goes to sleep and comes back out. In addition, WSL2 has no systemd
-and so cannot run Linux-native time sync easily.
-
-While this can all be solved with the use of 3rd-party software, I don't want to be
-responsible for someone losing money because time was off.
-
-If you know enough to get Windows 10 and WSL2 time sync stable, you likely also know enough
-to run a Linux server in the first place.
+The challenges inherent in running on Windows 10 are easier to solve when using the Windows-native
+versions of the clients, rather than wrapping Docker around them.

README.md

@@ -93,7 +93,8 @@ Please take a look.
 ## Step 1: Install prerequisites
 
 You will need git, docker, and docker-compose. This should work on Linux, possibly MacOS.
-Running a node on Windows 10 is highly discouraged because of time synchronization concerns.
+Running a node via this project on Windows 10 is [highly discouraged](WINDOWS.md) because of time
+synchronization concerns.
 Please see [prerequisites](PREREQUISITES.md) and then come back here.
 
 ## Step 2: Choose a client, initial setup
@@ -111,8 +112,7 @@ Please see [setup instructions](SETUP.md) and then come back here.
 Build all required images. If building from source, this may take 20-30 minutes. Assuming
 you cloned the project into `eth2-docker`:
 ```
-cd ~/eth2-docker
-sudo docker-compose build
+cd ~/eth2-docker && sudo docker-compose build
 ```
 
 ## Step 4: Create an eth2 wallet and validator keystore and deposit files
@@ -121,7 +121,16 @@ You will deposit eth to the deposit contract, and receive locked eth2 in turn.<b
 > **Vital** [RECOMMENDATIONS.md](RECOMMENDATIONS.md) has comments on key security. If you haven't
 read these yet, please do so now. You need to know how to guard your keystore password
 and your seed phrase (mnemonic). **Without the mnemonic, you will be unable to withdraw your funds
-in phase 2. You need the seed phrase or your eth is gone forever.**
+after the "merge", also called phase 1.5 of Ethereum 2.0. You need the seed phrase or your eth is gone forever.**
+
+> You can create the keys using eth2-docker. For mainnet, you may want to create
+> the keys on a machine that is not connected to the Internet, and will be wiped
+> afterwards. This can be done by downloading [eth2.0-deposit-cli](https://github.com/ethereum/eth2.0-deposit-cli)
+> directly and copying it to that machine, or by fetching the eth2-docker project
+> on that machine, then disconnecting it from Internet. After you've created
+> keys, you'd move them off the machine, wipe the machine used to create them,
+> copy them to the machine the node will run on, and continue from
+> "You brought your own keys", below.
 
 Make sure you're in the project directory, `cd ~/eth2-docker` by default.
 
@@ -196,7 +205,10 @@ marked offline if your validator is activated before the beacon syncs.
 
 Once you are ready, you can send eth to the deposit contract by using
 the `.eth2/validator_keys/deposit_data-TIMESTAMP.json` file at the [Pyrmont launchpad](https://pyrmont.launchpad.ethereum.org/)
-or [Mainnet launchpad](https://launchpad.ethereum.org).
+or [Mainnet launchpad](https://launchpad.ethereum.org). This file was created in Step 4.
+
+> You can transfer files from your node to a machine with a browser using scp. A graphical
+> tool such as WinSCP will work, or you can use [command line scp](https://linuxize.com/post/how-to-use-scp-command-to-securely-transfer-files/).
 
 ## Step 8: Grafana Dashboards
 
@@ -208,13 +220,17 @@ A baseline set of dashboards has been included.
 - [Nimbus Dashboard JSON](https://raw.githubusercontent.com/status-im/nimbus-eth2/master/grafana/beacon_nodes_Grafana_dashboard.json)
 - [Teku Dashboard](https://grafana.com/api/dashboards/12199/revisions/1/download)
 
-- Connect to http://YOURSERVERIP:3000/, log in as admin/admin, set a new password
+Connect to http://YOURSERVERIP:3000/, log in as admin/admin, and set a new password.
+> Grafana runs over http without encryption, which is not secure. Do not expose the Grafana port to the Internet. You can
+> use [SSH tunneling](https://www.howtogeek.com/168145/how-to-use-ssh-tunneling/) to reach Grafana securely over the Internet.
 
-In order to load Dashboards for other clients, follow these instructions
-
-- Import a Dashboard. Click on the + icon on the left, choose "Import". Copy/paste JSON code from one of the client dashboard links below (click anywhere inside the page
-the link gets you to, use Ctrl-a to select all and Ctrl-C to copy), click "Load", choose the "prometheus" data source you just configured, click "Import".
+In order to load other Dashboards, follow these instructions.
 
+- Click on the + icon on the left, choose "Import".
+- Copy/paste JSON code from the Raw github page of the Dashboard you chose - click anywhere inside the page, use Ctrl-A to select all and Ctrl-C to copy
+- Click "Load"
+- If prompted for a data source choose the "prometheus" data source
+- Click "Import".
 
 ## Step 9: Autostart the client on boot
 
@@ -267,21 +283,25 @@ show you how to.
 
 Please see the [changelog](CHANGELOG.md) to see what changed between versions.
 
-### The project itself
+### The eth2-docker tool itself
 
 Inside the project directory, run:<br />
 `git pull`
 
 Then `cp .env .env.bak` and `cp default.env .env`, and set variables inside `.env`
 the way you need them, with `.env.bak` as a guide.
 
-### Eth1
+### The client "stack"
 
-If not building from source, run:<br />
-`sudo docker-compose build --pull eth1`
+If you are using binary build files - the default - you can update everything
+in the client "stack" with `sudo docker-compose build --pull`. If you
+run shared components in a different directory, such as eth1, traefik, or portainer,
+you'd `cd` into those directories and run the command there.
 
-Else, if building from source:<br />
-`sudo docker-compose build --no-cache eth1`
+### Eth1
+
+Run:<br />
+`sudo docker-compose build --no-cache --pull eth1`
 
 Then stop, remove and start eth1:<br />
 `sudo docker-compose stop eth1 && sudo docker-compose rm eth1`<br />
@@ -291,17 +311,14 @@ Then stop, remove and start eth1:<br />
 
 Beacon and validator client share the same image for most clients, we only need to rebuild one.
 
-If not building from source, run:<br />
-`sudo docker-compose build --pull beacon`
-
-Else, if building from source:<br />
-`sudo docker-compose build --no-cache beacon`
+Run:<br />
+`sudo docker-compose build --no-cache --pull beacon`
 
 For Prysm, also run:<br />
-`sudo docker-compose build --pull validator`
+`sudo docker-compose build --no-cache --pull validator`
 
-Or, if building from source:<br />
-`sudo docker-compose build --no-cache validator`
+And if using the Prysm slasher, run:<br />
+`sudo docker-compose build --no-cache --pull slasher`
 
 Then restart the client:<br />
 `sudo docker-compose down && sudo docker-compose up -d eth2`

SETUP.md

@@ -50,6 +50,7 @@ Please choose:
   * geth
   * nethermind - testing only, DB corruption observed on goerli and mainnet
   * openethereum - testing only, DB corruption observed on mainnet
+  * besu - testing for now, I have not solved interop issues with Lighthouse
   * 3rd-party
 * Whether to run a slasher (experimental for Prysm)
 * Whether to run a grafana dashboard for monitoring
@@ -78,29 +79,30 @@ has that functionality built-in.
 > permissions errors during use.
 
 - Set the `COMPOSE_FILE` entry depending on the client you are going to run,
-and with which options. See below for available compose files
+and with which options. See below for available compose files. Think of this as
+blocks you combine: One ethereum 2 client, optionally one ethereum 1 node, optionally reporting.
 - If you are going to use a 3rd-party provider as your eth1 chain source, set `ETH1_NODE` to that URL.
-  See [how to create your own Infura account](https://status-im.github.io/nimbus-eth2/infura-guide)
+  Look into [Alchemy](https://alchemyapi.io) or see [how to create your own Infura account](https://status-im.github.io/nimbus-eth2/infura-guide)
 - Adjust ports if you are going to need custom ports instead of the defaults. These are the ports
-exposed to the host, and for everything but Grafana to the Internet via your firewall/router
-- Set the `NETWORK` variable to either "mainnet" or a test network such as "medalla"
+exposed to the host, and for the P2P ports to the Internet via your firewall/router.
+- Set the `NETWORK` variable to either "mainnet" or a test network such as "pyrmont"
 - If using geth as the eth1 node, comment out the `GETH1_NETWORK` variable, to use the main net, or set it to a test network such as "--goerli",
   with the two dashes.
-- With other eth1 nodes, the `ETH1_NETWORK` variable serves the same function. It can be set to `mainnet` to use the main eth1 network with
-Nethermind, or `ethereum` to use the main eth1 network with OpenEthereum.
-- Set the `GRAFFITI` string if you want a specific string
+- With other eth1 nodes, the `ETH1_NETWORK` variable serves the same function. It can be set to `mainnet` to use the main eth1 network.
+- Set the `GRAFFITI` string if you want a specific string.
 
 ### Client compose files
 
 Set the `COMPOSE_FILE` string depending on which client you are going to use. Add optional services like
-openethereum with `:` between the file names.
+geth with `:` between the file names.
 - `lh-base.yml` - Lighthouse
 - `prysm-base.yml` - Prysm
 - `teku-base.yml` - Teku
 - `nimbus-base.yml` - Nimbus
 - `geth.yml` - local geth eth1 chain node
 - `nm.yml` - local nethermind eth1 chain node - testing only, DB corruptionb observed on goerli and mainnet
 - `oe.yml` - local openethereum eth1 chain node - testing only, DB corruption observed on mainnet
+- `besu.yml` - local besu eth1 chain mode - testing for now, I have not solved interop with the Ethereum 2.0 client
 - `eth1-shared.yml` - makes the RPC port of the eth1 node available from the host, for using the eth1 node with other nodes or with Metamask. **Not encrypted**, do not expose to Internet.
 - `eth1-standalone.yml` - like eth1-shared but for running *just* eth1, instead of running it alongside a beacon node in the same "stack". Also not encrypted, not meant for a fully distributed setup quite yet.
 - `prysm-slasher.yml` - Prysm experimental Slasher which helps secure the chain and may result in additional earnings. The experimental slasher can lead to missed attestations do to the additional resource demand.
@@ -110,7 +112,7 @@ openethereum with `:` between the file names.
 - `nimbus-grafana.yml` - grafana dashboard for Nimbus
 - `teku-grafana.yml` - grafana dashboard for Teku
 
-For example, Lighthouse with local openethereum and grafana:
+For example, Lighthouse with local geth and grafana:
 `COMPOSE_FILE=lh-base.yml:geth.yml:lh-grafana.yml`
 
 > See [WEB](WEB.md) for notes on using the experimental Prysm Web UI
@@ -120,29 +122,39 @@ is in use, their own eth1 node. This is perfect for running a single client, or
 clients each in their own directory.
 
 If you want to run multiple isolated clients, just clone this project into a new directory for
-each. This is great for running medalla and zinken in parallel, for example.
+each. This is great for running testnet and mainnet in parallel, for example.
 
-### `SLASHER`   
-Running [slasher](https://docs.prylabs.network/docs/prysm-usage/slasher/) is an optional client compose file, but helps secure the chain and may result in additional earnings.
-```
-Slasher can be a huge resource hog during times of no chain finality, which can manifest as massive RAM usage. Please make sure you understand the risks of this, especially if you want high uptime for your beacon nodes. Slasher places significant stress on beacon nodes when the chain has no finality, and might be the reason why your validators are underperforming if your beacon node is under this much stress.
-```
+### Prysm Slasher   
+Running [slasher](https://docs.prylabs.network/docs/prysm-usage/slasher/) is an optional client compose file, but helps secure the chain and may result in additional earnings,
+though the chance of additional earnings is low in phase 0 as whistleblower rewards have not been implemented yet.
+
+> Slasher can be a huge resource hog during times of no chain finality, which can manifest as massive RAM usage. Please make sure you understand the risks of this, 
+> especially if you want high uptime for your beacon nodes. Slasher places significant stress on beacon nodes when the chain has no finality, and might be the reason
+> why your validators are underperforming if your beacon node is under this much stress.
 
 ## Firewalling
 
-You'll want to forward ports to the services of your eth2 node, and on Linux, enable a host firewall.
+You'll want to enable a host firewall. You can also forward the P2P ports of your eth1 and eth2
+nodes for faster peer acquisition.
+
 These are the relevant ports. docker will open eth2 node ports and the grafana port automatically,
 please make sure the grafana port cannot be reached directly. If you need to get to grafana remotely,
-an SSH tunnel is a good choice.
-Ports that I mention should be "Open to Internet" need to be either forwarded
+an [SSH tunnel](https://www.howtogeek.com/168145/how-to-use-ssh-tunneling/) is a good choice.
+
+Ports that I mention can be "Open to Internet" can be either forwarded
 to your node if behind a home router, or allowed in via the VPS firewall.
 
-- 30303 tcp/udp - local eth1 node, geth or openethereum or nethermind. Open to Internet.
-- 9000 tcp/udp - Lighthouse beacon node. Open to Internet.
-- 13000/tcp - Prysm beacon node. Open to Internet.
-- 12000/udp - Prysm beacon node. Open to Internet.
-- 9000 tcp/udp - Teku beacon node. Open to Internet. Note this is the same default port as Lighthouse.
-- 9000 tcp/udp - Nimbus beacon node. Open to Internet. Note this is the same default port as Lighthouse.
+> Opening the P2P ports to the Internet is optional. It will speed up peer acquisition, which
+> can be helpful. To learn how to forward your ports in a home network, first verify
+> that you are [not behind CGNAT](https://winbuzzer.com/2020/05/29/windows-10-how-to-tell-if-your-isp-uses-carrier-grade-nat-cg-nat-xcxwbt/).
+> Then look at [port-forwarding instructions](https://portforward.com/) for your specific router/firewall. 
+
+- 30303 tcp/udp - local eth1 node P2P. Open to Internet.
+- 9000 tcp/udp - Lighthouse beacon node P2P. Open to Internet.
+- 13000/tcp - Prysm beacon node P2P. Open to Internet.
+- 12000/udp - Prysm beacon node P2P. Open to Internet.
+- 9000 tcp/udp - Teku beacon node P2P. Open to Internet. Note this is the same default port as Lighthouse.
+- 9000 tcp/udp - Nimbus beacon node P2P. Open to Internet. Note this is the same default port as Lighthouse.
 - 3000/tcp - Grafana. **Not** open to Internet, allow locally only. It is insecure http.
 - 22/tcp - SSH. Only open to Internet if this is a remote server (VPS). If open to Internet, configure
   SSH key authentication.
@@ -200,7 +212,7 @@ file when running `ls ~/.ssh`.
 ### Create an SSH key pair
 
 Create a key if you need to, or if you don't have `id_ed25519.pub` but prefer that cipher:<br />
-`ssh-keygen -t ed25519`
+`ssh-keygen -t ed25519`. Set a strong passphrase for the key.
 > Bonus: On Linux, you can also include a timestamp with your key, like so:<br />
 > `ssh-keygen -t ed25519 -C "$(whoami)@$(hostname)-$(date -I)" -f ~/.ssh/id_ed25519`
 
@@ -225,13 +237,16 @@ And paste in the public key.
 ### Test login and turn off password authentication
 
 Test your login. `ssh user@serverIP` from your client's MacOS/Linux Terminal or Windows Powershell should log you in
-directly without prompting for a password.
+directly, prompting for your key passphrase, but not the user password.
+
+If you are still prompted for a password, resolve that first. Your ssh client should show you errors in that case. You
+can run `ssh -v user@serverIP` to get more detailed output on what went wrong.
 
-If you are still prompted for a password, resolve that first. Your ssh client should show you errors in that case.
 On Windows 10 in particular, if the ssh client complains about the "wrong permissions" on the `.ssh` directory or
 `.ssh/config` file, go into Explorer, find the `C:\Users\USERNAME\.ssh` directory, edit its Properties->Security, click
 Advanced, then make your user the owner with Full Access, while removing access rights to anyone else, such as SYSTEM
-and Administrators. That should solve the issues the OpenSSH client had.
+and Administrators. Check "Replace all child object permissions", and click OK. That should solve the issues the
+OpenSSH client had.
 
 Lastly, once key authentication has been tested, turn off password authentication. On your Linux server:<br />
 `sudo nano /etc/ssh/sshd_config`

WINDOWS.md

@@ -0,0 +1,28 @@
+# Windows, and its challenges
+
+Windows may seem like an "easy button". For eth2-docker, it is anything but, and even for
+other ways of running a client, there are multiple challenges. They can all be overcome,
+and, I have yet to see a comprehensive document that addresses all of them.
+
+Things to think about with Windows:
+
+- Time sync is off by (up to) a second out of the box in Windows, and can be so bad in WSL2 you can't attest. 
+  Solve w/ 3rd party in Windows. Check via https://time.is
+- Time sync in WSL 2 worse, which comes into play when using Docker Desktop. I've heard "I sync it manually" - not sustainable for 2+ years.
+- Running as a service. It's 100% doable, and, not terribly well documented for this use case right now. How to differs between Docker Desktop
+  and other ways of running. You'd be looking into Windows Task Scheduler.
+- Client diversity. Prysm does Windows-native, Lighthouse will soon. The other two would rely on a setup that runs them in Docker Desktop and WSL2,
+  thus Linux, afaik.
+- eth1 node - Prysm sidesteps this with 3rd-party, and Geth runs natively. For other clients, you're looking at Docker Desktop again.
+- Remote administration - SSH? RDP? If RDP, do you need a "proper" cert to encrypt?
+- If using Docker Desktop and WSL2, consider that WSL2 runs as a virtual network. Work-around is to run a Powerscript that addresses needed port-forwarding.
+- I've had Docker Desktop fail to start, rarely. It appears to be related to when there is an update available, which would typically be the case
+  upon restart of the host. This needs to be solved, I am not sure how.
+- Let's harp on "running as a service" one more time: By default, Docker Desktop doesn't start until the user logs in. You need to
+  be up 24/7 and your node won't start until someone logs in. Which can be solved - the instructions looked involved. You also want security, which
+  means auto login of user is not a solution here.
+
+The best bet for Windows is likely to run Prysm or Lighthouse natively, with Geth, some way of starting them as a service without a user needing to be logged
+in (Task Scheduler), and 3rd-party software to improve time sync.
+
+That is absolutely doable.

lighthouse/Dockerfile.source

@@ -3,7 +3,7 @@ FROM rust:latest as builder
 
 ARG BUILD_TARGET
 
-RUN apt-get update && apt-get install -y git gcc g++ make cmake pkg-config libssl-dev
+RUN apt-get update && apt-get install -y git gcc g++ make cmake pkg-config libssl-dev patch --no-install-recommends
 
 WORKDIR /usr/src
 RUN bash -c "git clone https://github.com/sigp/lighthouse.git && cd lighthouse && git config advice.detachedHead false && git fetch --all --tags && git checkout ${BUILD_TARGET} && make"