Skip to content

Commit

Permalink
Moved assemblies to own common folder
Browse files Browse the repository at this point in the history
Updated README.md
Improved configuration testing
Removed unused certificate files after generating
Removed unused genesis state files after generating
singleCurrency and dualCurrency presets.
nemesis seed is shared across the different nodes avoiding duplicated files.
feat: allowing custom alpha tag for npm packages
  • Loading branch information
fboucquez authored and Fernando committed Nov 26, 2021
1 parent 420e2c1 commit 788c803
Show file tree
Hide file tree
Showing 154 changed files with 3,365 additions and 21,108 deletions.
1 change: 0 additions & 1 deletion .prettierignore
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
/test/reports/**/*.json
89 changes: 55 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,7 @@ Symbol CLI tool that allows you creating, configuring and running Symbol's c
* [Key benefits:](#key-benefits)
* [Concepts](#concepts)
* [Requirements](#requirements)
* [Usage](#usage)
* [General Usage](#general-usage)
* [Installation](#installation)
* [Wizard](#wizard)
* [E2E Testing support](#e2e-testing-support)
* [Development](#development)
Expand All @@ -27,12 +26,7 @@ Symbol CLI tool that allows you creating, configuring and running Symbol's c

# Why this tool?

This tool has been created to address the problems defined in Symbol's [NIP11](https://github.com/nemtech/NIP/blob/main/NIPs/nip-0011.md).

It replaces:

- [catapult-service-bootstrap](https://github.com/nemtech/catapult-service-bootstrap)
- [symbol-testnet-bootstrap](https://github.com/nemgrouplimited/symbol-service-bootstrap)
This tool has been created to address the original problems defined in Symbol's [NIP11](https://github.com/nemtech/NIP/blob/main/NIPs/nip-0011.md).

# Key benefits:

Expand All @@ -58,23 +52,26 @@ Yaml files that define the configuration and layout of the network and nodes. It
Presets are defined at 4 levels from general to specific:

- Shared: Default configurations for all the networks.
- Network: It defines the main preset of a given network, example: `bootstrap` or `testnet`.
- Assembly: It defines a modification of a network, example: `testnet peer`, `tesnet dual`, `testnet api`. Assembly is required for some networks (like `testnet`).
- Network: It defines the configuration of a given network.
- Assembly: It defines a modification of a network selecting the services that the node will run.
- Custom: A user provided yml file (`--customPreset` param) that could override some or all properties in the out-of-the-box presets.

Properties in each file override the previous values (by object deep merge).

### Out-of-the-box presets and assemblies:
### Network Presets:

- [`mainnet`](presets/mainnet/network.yml): Used to created nodes' connected to Symbol's Mainnet network. The [nemesis block](presets/mainnet/seed/00000) is copied over.
- [`testnet`](presets/testnet/network.yml): Used to created nodes' connected to Symbol's Testnet network. The [nemesis block](presets/testnet/seed/00000) is copied over.
- [`dualCurrency`](presets/dualCurrency/network.yml): Used to create new private networks with dual currency configuration, network and harvest currencies. Nemesis block is generated.
- [`singleCurrency`](presets/singleCurrency/network.yml): Used to create private/user networks with a single network currency configuration, just main network currency. Nemesis block is generated.

- `-p bootstrap`: A [multinode](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/bootstrap/network.yml) local network. A network with 1 mongo database, 2 peers, 1 api and 1 rest gateway. Nemesis block is generated.
- `-p bootstrap -a light`: A [light](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/bootstrap/assembly-light.yml) local network. It's a version of bootstrap preset with 1 mongo database, 1 dual peer and 1 rest gateway. Great for faster light e2e automatic testing. Nemesis block is generated.
- `-p bootstrap -a full`: A [full](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/bootstrap/assembly-full.yml) local network. The bootstrap preset plus 1 wallet, 1 explorer and 1 faucet. Great for demonstration purposes. Nemesis block is generated.
- `-p testnet -a peer`: A [harvesting](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/testnet/assembly-peer.yml) peer node that connects to the current public [testnet](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/testnet/network.yml). [Nemesis block](https://github.com/nemtech/symbol-bootstrap/tree/main/presets/testnet/seed/00000) is copied over.
- `-p testnet -a api`: A [api](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/testnet/assembly-api.yml) peer node that connects to the current public [testnet](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/testnet/network.yml) running its own mongo database and rest gateway. [Nemesis block](https://github.com/nemtech/symbol-bootstrap/tree/main/presets/testnet/seed/00000) is copied over.
- `-p testnet -a dual`: A [dual](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/testnet/assembly-dual.yml) haversting peer node that connects to the current public [testnet](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/testnet/network.yml) running its own mongo database and rest gateway. [Nemesis block](https://github.com/nemtech/symbol-bootstrap/tree/main/presets/testnet/seed/00000) is copied over.
- `-p mainnet -a peer`: A [harvesting](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/mainnet/assembly-peer.yml) peer node that connects to the current public [mainnet](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/mainnet/network.yml). [Nemesis block](https://github.com/nemtech/symbol-bootstrap/tree/main/presets/mainnet/seed/00000) is copied over.
- `-p mainnet -a api`: A [api](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/mainnet/assembly-api.yml) peer node that connects to the current public [mainnet](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/mainnet/network.yml) running its own mongo database and rest gateway. [Nemesis block](https://github.com/nemtech/symbol-bootstrap/tree/main/presets/mainnet/seed/00000) is copied over.
- `-p mainnet -a dual`: A [dual](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/mainnet/assembly-dual.yml) haversting peer node that connects to the current public [mainnet](https://github.com/nemtech/symbol-bootstrap/blob/main/presets/mainnet/network.yml) running its own mongo database and rest gateway. [Nemesis block](https://github.com/nemtech/symbol-bootstrap/tree/main/presets/mainnet/seed/00000) is copied over.
### Assemblies:

- [`dual`](presets/assemblies/assembly-dual.yml): A standard dual node that contains 1 mongo database. 1 api node, 1 rest gateway, and 1 broker.
- [`peer`](presets/assemblies/assembly-peer.yml): A standard peer only node that contains 1 peer node.
- [`api`](presets/assemblies/assembly-api.yml): A standard api node that contains 1 mongo database. 1 api node, 1 rest gateway, and 1 broker.
- [`demo`](presets/assemblies/assembly-demo.yml): A dual node that contains an explorer, web wallet and faucet for test and demonstration purposes.
- [`multinode`](presets/assemblies/assembly-multinode.yml): An special assembly that contains 1 api node and 2 peer only nodes. This assembly is for testing, it showcases how a private network with 3 nodes runs.

### Custom preset:

Expand All @@ -84,7 +81,9 @@ Custom presets give Symbol Bootstrap its versatility. Check out the custom prese

## Target:

The folder where the generated config, docker files and data are stored. The folder structure is:
The folder where the generated config, docker files and data are stored.

The folder structure is:

- `./preset.yml`: the final generated preset.yml that it's used to configure bootstrap, the nodes, docker, etc.
- `./addresses.yml`: randomly generated data that wasn't provided in the preset. e.g.: SSL keys, nodes' keys, nemesis accounts, generation hash seed, etc.
Expand All @@ -93,20 +92,18 @@ The folder where the generated config, docker files and data are stored. The fol
- `./nemesis`: The folder used to hold the nemesis block. Block 1 data is generated via `nemgen` tool for new networks. For existing network, it is copied over.
- `./databases`: the location where the mongo data is stored for the different database instances.
- `./docker`: the generated docker-compose.yml, mongo init scripts and server basic bash scripts.
- `./explorers`: the generated explorer configuration.
- `./wallets`: the generated wallet configuration.
- `./reports`: the location of the generated reports.

Note: **The target folder should not be manually modified**. This tool may override any file in the target folder when doing upgrades. Any custom configuration should be provided via a custom preset. Check out the custom preset [guides](docs/presetGuides.md)!

# Requirements

- Node 12.0.0+
- Docker 18.3.0+
- Docker Compose 1.25.0+

Validate your environment by running:

```
symbol-bootstrap verify
```

Check your user can run docker without sudo:

```
Expand All @@ -121,7 +118,7 @@ Got permission denied while trying to connect to the Docker daemon socket at uni

Please follow this [guide](https://www.digitalocean.com/community/questions/how-to-fix-docker-got-permission-denied-while-trying-to-connect-to-the-docker-daemon-socket).

# Usage
# Installation

It's recommended to run the commands from en empty working dir.

Expand All @@ -148,20 +145,24 @@ USAGE
```
<!-- usagestop -->

# General Usage
Validate your environment by running:

```
symbol-bootstrap verify
```

The general usage would be:

```
symbol-bootstrap config -p bootstrap
symbol-bootstrap config -p testnet -a dual
symbol-bootstrap compose
symbol-bootstrap run
```

You can aggregate all these commands with this one liner:

```
symbol-bootstrap start -p bootstrap
symbol-bootstrap start -p testnet -a dual
```

If you need to start fresh, you many need to sudo remove the target folder (docker volumes dirs may be created using sudo). Example:
Expand All @@ -170,6 +171,26 @@ If you need to start fresh, you many need to sudo remove the target folder (dock
sudo rm -rf ./target
```

## Examples

Network presets and assemblies can be combined to generate different types of nodes. Some examples:

- `$ symbol-bootstrap start -p mainnet -a dual -c customPreset.yml`
- `$ symbol-bootstrap start -p testnet -a peer -c customPreset.yml`
- `$ symbol-bootstrap start -p testnet -a demo -c customPreset.yml`
- `$ symbol-bootstrap start -p dualCurrency -a multinode -c customPreset.yml`
- `$ symbol-bootstrap start -p dualCurrency -a demo -c customPreset.yml`
- `$ symbol-bootstrap start -p dualCurrency -a dual -c customPreset.yml`
- `$ symbol-bootstrap start -p singleCurrency -a multinode -c customPreset.yml`
- `$ symbol-bootstrap start -p singleCurrency -a demo -c customPreset.yml`

Although some combinations can be done, they may not be really useful. Examples are running `mainnet` and `testnet` combined the `demo` or `multinode` assemblies.

A custom network preset file can also be provided. This is useful when you have your own custom Symbol network, and you want other nodes to join.
For this case, you provide your own `networkPreset.yml` and nemesis feed folder. The node admin can then run:

- `$ symbol-bootstrap start -p customNetworkPreset.yml -a dual -c customNodePreset.yml`

# Wizard

If this is your first time creating a node, it's recommended to use the Wizard. Just follow the instructions:
Expand All @@ -183,7 +204,7 @@ symbol-bootstrap wizard
One use case of this CLI is client E2E testing support. If you are coding a Symbol client, you (Travis or Jenkins) can run e2e tests like:

```
symbol-bootstrap start -p bootstrap --detached
symbol-bootstrap start -p dualCurrency -a dual -c my_custom_preset.yml --detached --healthCheck
YOUR TEST (e.g: npm run test, gradle test, selenium etc.)
symbol-bootstrap stop
```
Expand All @@ -208,7 +229,7 @@ Your `package.json` can look like this:
scripts": {
...
"clean-network": "symbol-bootstrap clean",
"run-network": "symbol-bootstrap start -c ./output/my_custom_preset.yml --detached --healthCheck",
"run-network": "symbol-bootstrap start -p dualCurrency -a dual -c my_custom_preset.yml --detached --healthCheck",
"run-stop": "symbol-bootstrap stop",
"integration-test": "....some mocha/jest/etc tests running against localhost:3000 network....",
"e2e": "npm run clean-network && npm run run-network && npm run integration-test && npm run stop-network",
Expand All @@ -224,7 +245,7 @@ npm run e2e

## Node client E2E via API:

Alternatively, you can use the [BootstrapService](https://github.com/nemtech/symbol-bootstrap/blob/main/src/service/BootstrapService.ts) facade to programmatically start and stop a server.
Alternatively, you can use the [BootstrapService](src/service/BootstrapService.ts) facade to programmatically start and stop a server.

Example:

Expand Down
6 changes: 4 additions & 2 deletions alpha.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@
set -e
. ./travis/node-functions.sh
VERSION="$(node_load_version)"
echo $VERSION
npm pack && /bin/bash travis/node-functions.sh node_publish_alpha
TAG=${1:-alpha}
echo "publishing $VERSION with tag $TAG"
npm pack
/bin/bash travis/node-functions.sh node_publish_alpha $TAG
npm version "$VERSION" --commit-hooks false --git-tag-version false
npm run style:fix
6 changes: 0 additions & 6 deletions cmds/clean-all.sh

This file was deleted.

4 changes: 0 additions & 4 deletions cmds/config-bootstrap.sh

This file was deleted.

4 changes: 4 additions & 0 deletions cmds/config-dualCurrency.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
set -e

symbol-bootstrap config -p dualCurrency -a multinode --report -t target/bootstrap $1
4 changes: 0 additions & 4 deletions cmds/ngrok-testnet.sh

This file was deleted.

5 changes: 0 additions & 5 deletions cmds/start-bootstrap-alpha.sh

This file was deleted.

3 changes: 0 additions & 3 deletions cmds/start-bootstrap-custom.sh

This file was deleted.

4 changes: 0 additions & 4 deletions cmds/start-bootstrap-freenodes.sh

This file was deleted.

4 changes: 0 additions & 4 deletions cmds/start-bootstrap-full.sh

This file was deleted.

3 changes: 0 additions & 3 deletions cmds/start-bootstrap-optin.sh

This file was deleted.

4 changes: 0 additions & 4 deletions cmds/start-bootstrap-repeat.sh

This file was deleted.

7 changes: 0 additions & 7 deletions cmds/start-bootstrap-rest.sh

This file was deleted.

4 changes: 0 additions & 4 deletions cmds/start-bootstrap-sudo.sh

This file was deleted.

5 changes: 0 additions & 5 deletions cmds/start-bootstrap.sh

This file was deleted.

3 changes: 3 additions & 0 deletions cmds/start-dualCurrency-custom.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
set -e
symbol-bootstrap start -u current -p dualCurrency -a multinode -c test/custom_preset.yml -t target/bootstrap-custom --password 1234 $1
4 changes: 4 additions & 0 deletions cmds/start-dualCurrency-demo.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
set -e

symbol-bootstrap start -p dualCurrency -a demo -t target/bootstrap --password 1234 -c test/full_preset.yml $1 $2 $3
4 changes: 4 additions & 0 deletions cmds/start-dualCurrency-freenodes.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
set -e

symbol-bootstrap start --reset -p dualCurrency -a multinode --customPreset ./test/freenodes_preset.yml --user current -t target/bootstrap-freenodes
4 changes: 4 additions & 0 deletions cmds/start-dualCurrency-repeat.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
docker stop $(docker ps -aq)
docker rm $(docker ps -aq)
symbol-bootstrap start -p dualCurrency -a multinode -c test/repeat_preset.yml -t target/bootstrap-repeat --password 1234 $1
5 changes: 5 additions & 0 deletions cmds/start-dualCurrency.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash
set -e

# docker rm -f $(docker ps -aq)
symbol-bootstrap start -p dualCurrency -a multinode -u current -t target/bootstrap -c test/custom_bootstrap.yml --password 1234 $1
4 changes: 0 additions & 4 deletions cmds/start-light.sh

This file was deleted.

5 changes: 5 additions & 0 deletions cmds/start-singleCurrency.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash
set -e

# docker rm -f $(docker ps -aq)
symbol-bootstrap start -p singleCurrency -u current -t target/singleCurrency -c test/custom_bootstrap.yml --password 1234 $1
Empty file modified cmds/stop-bootstrap.sh
100755 → 100644
Empty file.
2 changes: 1 addition & 1 deletion config/rest-gateway/rest.json.mustache
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"network": {
"name": "{{{networkName}}}",
"description": "{{{description}}}"
"description": "{{{networkDescription}}}"
},

"port": 3000,
Expand Down
Loading

0 comments on commit 788c803

Please sign in to comment.