Clarifications, support for experimental Prysm Web UI

CHANGELOG.md

@@ -15,6 +15,10 @@ directoy (`cd ~/eth2-docker` by default):
 * !! If coming from Prysm alpha.29 or earlier, make changes as per notes for [v0.1.7](#v017-2020-10-15)
 * `sudo docker-compose up -d eth2`
 
+## v0.1.7.4 2020-10-29
+
+* Support experimental Prysm Web UI
+
 ## v0.1.7.3 2020-10-27
 
 * Prysm change to remove creation of new protection DB, Prysm no longer has this flag

PREREQUISITES.md

@@ -11,11 +11,15 @@ expected to work on MacOS.
 > will run your node on. The machine you use to connect *to* the Linux server
 > only requires an SSH client.
 
+Run these one line at a time:
 ```
 sudo apt update && sudo apt dist-upgrade
-sudo apt install docker docker-compose git
+sudo apt install -y docker docker-compose git
 ```
 
+You know it was successful when you saw messages scrolling past that install git,
+docker and docker-compose.
+
 Other distributions are expected to work as long as they support
 git, docker, and docker-compose.
 

README.md

@@ -1,4 +1,4 @@
-# eth2-docker v0.1.7.2
+# eth2-docker v0.1.7.4
 Unofficial and experimental docker build instructions for eth2 clients
 
 ## Acknowledgements
@@ -28,6 +28,9 @@ Currently supported optional components:
 This can be a local geth in archive mode via ws://, or a 3rd-party provider via wss://. 
 Please see [SETUP](SETUP.md) for details.
 
+Please see [WEB](WEB.md) for experimental Web UI support on Prysm, and use the Web instead
+of validator-import to import keys.
+
 # USAGE
 
 ## Before you start
@@ -100,6 +103,9 @@ They go into `.eth2/validator_keys` in this project directory, not directly unde
 
 **Warning** Import your validator key(s) to only *one* client.
 
+> If you want to use the experimental [Prysm Web UI](WEB.md), use it to
+> import keys and not this command-line process.
+
 Import the validator key(s) to the validator client:
 
 `sudo docker-compose run validator-import`

SETUP.md

@@ -29,11 +29,11 @@ From a terminal and logged in as the user you'll be using from now on, and assum
 you'll be storing the project in your `$HOME`, run:
 
 ```
-cd ~
-git clone https://github.com/eth2-educators/eth2-docker.git
-cd eth2-docker
+cd ~ && git clone https://github.com/eth2-educators/eth2-docker.git && cd eth2-docker
 ```
 
+You know this was successful when your prompt shows `user@host:~/eth2-docker`
+
 > Note: All work will be done from within the `~/eth2-docker` directory.
 > All commands that have you interact with the "dockerized" client will
 > be carried out from within that directory.
@@ -93,13 +93,15 @@ geth with `:` between the file names.
 - `geth.yml` - local geth eth1 chain node
 - `geth-archive.yml` - local geth node in full archive mode, required for Nimbus
 - `lh-grafana.yml` - grafana dashboard for Lighthouse
-- `prysm-grafana.yml` - grafana dashboard for Prysm
+- `prysm-grafana.yml` - grafana dashboard for Prysm, as well as experimental Web UI
 - `nimbus-grafana.yml` - grafana dashboard for Nimbus
 - `teku-grafana.yml` - grafana dashboard for Teku
 
 For example, Lighthouse with local geth and grafana:
 `COMPOSE_FILE=lh-base.yml:geth.yml:lh-grafana.yml`
 
+> See [WEB](WEB.md) for notes on using the experimental Prysm Web UI
+
 In this setup, clients are isolated from each other. Each run their own validator, and if geth
 is in use, their own geth. This is perfect for running a single client, or multiple isolated
 clients each in their own directory.

WEB.md

@@ -0,0 +1,73 @@
+# Prsym Web UI
+
+The Prysm Web UI is new and still experimental. It is designed to be accessed locally, not remotely,
+which means an [SSH tunnel](https://www.howtogeek.com/168145/how-to-use-ssh-tunneling/) is required to access it.
+
+The `prysm-grafana.yml` file, specified in the `COMPOSE_FILE` variable inside `.env`, enables both Grafana
+and Web UI.
+
+## Prepare the validator client
+
+The Web UI will be used to import keys and create a wallet, but we also need the password for this
+wallet while starting the validator. To get around this chicken-and-egg problem, you can either
+edit `prysm-base.yml` and choose to provide the password whenever the validator starts, or run
+`sudo docker-compose run validator-import` now and choose the wallet password you will use during
+the Web UI Wallet Creation.
+
+> This password needs to be at least 8 characters long and contain both a number and a special
+> character. The script that stores the password here does not enforce that, but the Web UI does.
+
+Either way, once you are done, run `sudo docker-compose up -d eth2` to start the Prysm beacon
+and validator.
+
+## Connect to the Web UI
+
+Assuming you will access the Web UI remotely, from a machine that is not running the node, you'll need
+to open an SSH connection and tunnel the ports used by the Web UI.
+
+Example ssh command:
+```
+ssh -L 7500:<host>:7500 -L 3500:<host>:3500 -L 8080:<host>:8080 -L 8081:<host>:8081 -L 3000:<host>:3000 <user>@<host>
+```
+
+where `<host>` is the name or IP address of the node.
+
+Placing this into an alias or shell script can make life easier.
+
+Once the SSH tunnel is open, in a browser, open `http://127.0.0.1:7500`. You'll be prompted for a web password,
+which doesn't yet exist, and there is an option to "Create a Wallet".
+
+> Note this is insecure http. Encrypting this connection is supported by Prysm, but not yet incorporated in
+> this project. Look into TLS keys if you wish to change the gRPC connections to be encrypted.
+
+# Import keys
+
+Assuming you have some `keystore-m` JSON files from `sudo docker-compose run deposit-cli` or some other way
+of creating Launchpad compatible keys, click on "Create a Wallet".
+
+> These files are in `.eth2/validator_keys` if you used the `deposit-cli` workflow. You'll want to
+> move them to the machine you are running the browser on.
+
+Choose to create an "Imported Wallet" and enter `/var/lib/prysm` as the wallet directory.
+
+Select the `keystore-m` file(s), Continue, provide the password to the keystore, and Continue.
+
+Set a web password. For security reasons this should be different from the web password. Continue.
+
+Set the wallet password.  If you chose to store the wallet password with the validator in a previous step,
+make sure it matches here: This is the step where you actually create the wallet with that password.
+
+Continue and you will find yourself inside the Web UI, which will show you the beacon syncing. Once sync is
+complete, you will also see validator information.
+
+# Optional: Verify that wallet password was stored correctly
+
+If you chose to start the validator with a stored wallet password, verify that it was stored
+correctly by running these commands, one at a time:
+
+```
+sudo docker-compose down && sudo docker-compose up -d eth2
+sudo docker-compose logs -f validator
+```
+
+You'll need to navigate to the root of the Web UI and log in again after the restart.

lh-base.yml

@@ -84,6 +84,28 @@ services:
       - --testnet
       - ${TESTNET}
       - --debug-level=${LOG_LEVEL}
+  validator-account:
+    restart: "no"
+    user: ${LOCAL_UID}:${LOCAL_UID}
+    image: lighthouse
+    build:
+      context: ./lighthouse
+      args:
+        - BUILD_TARGET=${LH_BUILD_TARGET}
+        - USER=${LH_USER}
+        - UID=${LOCAL_UID}
+    volumes:
+      - lhvalidator-data:/var/lib/lighthouse
+      - ${DEPCLI_LOCALDIR}:/interchange
+    entrypoint:
+      - lighthouse
+      - account
+      - validator
+      - --datadir
+      - /var/lib/lighthouse
+      - --testnet
+      - ${TESTNET}
+      - --debug-level=${LOG_LEVEL}
   deposit-cli:
     restart: "no"
     user: ${LOCAL_UID}:${LOCAL_UID}

prysm-base.yml

@@ -60,6 +60,7 @@ services:
       - ${LOG_LEVEL}
       - --blst
       - --${TESTNET}
+      - --accept-terms-of-use
       # If you chose not to store the wallet password during import, comment out the two following lines
       - --wallet-password-file
       - /var/lib/prysm/password.txt

prysm-grafana.yml

@@ -1,23 +1,32 @@
 version: "3"
 services:
   beacon:
-    expose:
-      - 3500
-      - 8080
+    ports:
+      - 3500:3500/tcp
+      - 8080:8080/tcp
     command:
       - --monitoring-host
       - 0.0.0.0
       - --grpc-gateway-host
       - 0.0.0.0
+      - --grpc-gateway-corsdomain
+      - http://127.0.0.1:7500
+      - --grpc-gateway-port
+      - "3500"
   validator:
     ports:
-      - 4242:4242/tcp
-    expose:
-      - 8081
+      - 7500:7500/tcp
+      - 8081:8081/tcp
     command:
       - --monitoring-host
       - 0.0.0.0
-#      - --web
+      - --web
+      - --grpc-gateway-host
+      - 0.0.0.0
+      - --grpc-gateway-corsdomain
+      - http://127.0.0.1:7500
+      - --grpc-gateway-port
+      - "7500"
   prometheus:
     restart: "${RESTART}"
     build:

prysm/create-prysm-validator-wallet.sh

@@ -1,13 +1,27 @@
 #!/bin/bash
 # This will be passed arguments that start the validator
-echo When asked for a wallet directory below, enter /var/lib/prysm
+while true; do
+  read -p "Will you import keys via the Web UI? (y/n) " yn
+  case $yn in
+    [Yy]* ) import=0; echo "Skipping import. If you choose to store the wallet password, use the one you created during Web UI wallet creation"; break;;
+    [Nn]* ) import=1; echo "Continuing to key import"; break;;
+    * ) echo "Please answer yes or no.";;
+  esac
+done
+
 echo
-"$@"
 
-if [ $? -ne 0 ]; then
-  exit 1;
+if [ $import -ne 0 ]; then
+  echo When asked for a wallet directory below, enter /var/lib/prysm
+  echo
+  "$@"
+
+  if [ $? -ne 0 ]; then
+    exit 1;
+  fi
+  echo
 fi
-echo
+
 echo Storing the wallet password in plain text will allow the validator to start automatically without user input.
 echo
 while true; do
@@ -20,9 +34,14 @@ while true; do
 done
 echo
 while true; do
-  read -sp "Please enter the 'New wallet password' you chose above: " password1
+  if [ $import -ne 0 ]; then
+    prompt="Please enter the 'New wallet password' you chose above : "
+  else
+    prompt="Please choose a wallet password, which you will then also provide during Web UI Wallet Creation: "
+  fi
+  read -sp "${prompt}" password1
   echo
-  read -sp "Please re-enter the 'New wallet password': " password2
+  read -sp "Please re-enter the wallet password: " password2
   if [ "$password1" == "$password2" ]; then
     break
   else