Update setup readme

README.md

@@ -8,10 +8,56 @@ All private keys used in the library are for testing only and not used in any pr
 
 ## Prerequisites
 
-1. bitcoind
-2. A fatjar of the federate node
-2. [OPTIONAL] HSM signer. (only needed if an hsmConfigs configuration is specified)
-3. `node` and `npm` latest LTS version
+### bitcoind
+- download
+  Current pipeline in jenkins is running bitcoind 0.17, 0.18.1 works for local execution.
+  - Go to https://bitcoin.org/en/release/v0.18.1 in this page there is the link to the download sources
+  https://bitcoincore.org/bin/bitcoin-core-0.18.1/
+  Select the *.tar.gz file according to your OS (for MacOS sillicon [bitcoin-0.18.1-osx64.tar.gz](https://bitcoincore.org/bin/bitcoin-core-0.18.1/bitcoin-0.18.1-osx64.tar.gz))
+    to use curl instead use `curl -O https://bitcoin.org/bin/bitcoin-core-0.18.1/bitcoin-0.18.1-osx64.tar.gz`
+  - Decompress and copy the path to the folder.
+  - run in terminal `sudo cp <path to bitcoin-core folder>/bin/bitcoin* /usr/local/bin`
+  - create a folder called bitcoindata in your workspace - this is the folder were the Bitcoin DB is going to be stored.
+  - run to test the bitcoind installation standalone `bitcoind -deprecatedrpc=generate -addresstype=legacy -regtest -printtoconsole -server -rpcuser=rsk -rpcpassword=rsk -rpcport=18332 -txindex -datadir=<path to bitcoindata> $@`
+
+### Federate node setup
+- install java the x86 version to be used with rossetta in case of ARM arch (1.8, 11 and 17 can be used).
+- A fatjar of the federate node can be used or follow the setup steps in the powpeg-node to run from scratch.
+
+
+### HSM 
+- Clone the HSM repository https://github.com/rootstock/hsm
+- Install python 2.7.16 from the python official site https://www.python.org/downloads/release/python-2716/
+- In terminal execute
+  - `pip install wheel`
+  - `pip install --no-cache secp256k1==0.13.2` (last version 0.14.0 is not working correctly with python 2)
+  - test the hsm1 by executing `python fedhm-mockup.py`
+  - follow the readme in hsm repository for hsm2 config.
+
+  The pipeline uses the same configuration as in `regtest.js` 
+  There is a `regtest.js.sample.hsm1only` that can be set as `regtest.js` to avoid usage of hsm2 if required.
+### NODE.JS
+- Install node.js latest LTS version (recommendation use [nvm](https://formulae.brew.sh/formula/nvm))
+
+### OPTIONAL (LOGS)
+
+In order to keep a log of each federate node a logback can be setup from the `regtest.js` file per each federate.
+There is sample in the logbacks folder that can be copied into each of the federate nodes.
+The sample name is `logback-fed1.xml.sample` and inside the change is in the line
+```
+<file>/PATH/TO/LOGS/fed1.log</file>
+```
+to determine where the logs are going to be written.
+Inside the `regtest.js` file the logbacks per federate node can be setup and mapped as
+```
+const federatesLogbackPath = '/PATH/TO/logbacks/'
+const customLogbackFile1 = federatesLogbackPath + 'logback-fed1.xml';
+const customLogbackFile2 = federatesLogbackPath + 'logback-fed2.xml';
+const customLogbackFile3 = federatesLogbackPath + 'logback-fed3.xml';
+const customLogbackFile4 = federatesLogbackPath + 'logback-fed4.xml';
+const customLogbackFile5 = federatesLogbackPath + 'logback-fed5.xml';
+``` 
+(The names of the logback files are suggestions)
 
 ## Running the tests
 
@@ -22,8 +68,14 @@ All private keys used in the library are for testing only and not used in any pr
   - Set `federate.classpath` to your local federate node fatjar. Additionally you can set `federate.configFile` to your local configuration files, if you need to test special conditions or your configuration files are located in a different path.
   - Set `federate.hsmConfigs.serverPath` to your local HSM emulator script.
   - Optionally specify `federate.hsmConfigs.keyPath` to your local keys, if you happen to be testing different keys or your keys are located in a different path.
-4. 
-	-  Run `npm test` to run testcases without interruption or `npm run test-fail-fast` to stop execution at first testcase failure.
+4. Run `npm test` to run testcases without interruption or `npm run test-fail-fast` to stop execution at first testcase failure.
+
+### Notes for regtest
+- `regtest.js` by default has the following mapping:
+    - federate1: key file (no hsm)
+    - federate2 and federate3: hsm1
+    - federate4 and federate5: hsm2 (modify using the same logic that in federate2 and 3 for changing to hsm1 in case of hsm2 not working as explained in the HSM setup section).
+
 
 ## Running the tests with a different configuration file
 

config/regtest.js.sample.hsm1only

@@ -0,0 +1,132 @@
+const path = require('path');
+// const version = 'FINGERROOT-5.2.0.0';
+const nodesConfigPath = 'config/node-configs/';
+const keysPathResolve = 'node-keys/';
+const keysPath = 'config/node-keys/';
+const classpath = 'PATH/TO/federate-node/build/libs/federate-node-' + version + '-all.jar';
+const hsmV1ServerPath = 'PATH/TO/hsm/hsm/fedhm-mockup.py';
+const hsmV2ServerPath = 'PATH/TO/hsm/hsm/v2/sim/0.1.3/sim';
+const federatesLogbackPath = '/PATH/TO/logbacks/'
+const customLogbackFile1 = federatesLogbackPath + 'logback-fed1.xml';
+const customLogbackFile2 = federatesLogbackPath + 'logback-fed2.xml';
+const customLogbackFile3 = federatesLogbackPath + 'logback-fed3.xml';
+const customLogbackFile4 = federatesLogbackPath + 'logback-fed4.xml';
+const customLogbackFile5 = federatesLogbackPath + 'logback-fed5.xml';
+
+module.exports = {
+  init: {
+    mineInitialBitcoin: true,
+    federatesLogbackFile: federatesLogbackPath
+  },
+  btc: { // Bitcoind configuration
+    rpcUser: 'test',
+    rpcPassword: 'test'
+  },
+  federate: [
+    { // Federate node configuration, can be an array to start multiple federate nodes
+        logbackFile: customLogbackFile1,
+        classpath: classpath,
+        configFile: nodesConfigPath + 'rsk-reg-1.conf',
+        publicKeys: {
+          btc: '0362634ab57dae9cb373a5d536e66a8c4f67468bbcfb063809bab643072d78a124',
+          rsk: '0362634ab57dae9cb373a5d536e66a8c4f67468bbcfb063809bab643072d78a124',
+          mst: '0362634ab57dae9cb373a5d536e66a8c4f67468bbcfb063809bab643072d78a124',
+        },
+        customConfig: {
+          'federator.signers.BTC.type': 'keyFile',
+          'federator.signers.BTC.path': path.resolve(__dirname, `${keysPathResolve}reg1.key`),
+          'federator.signers.RSK.type': 'keyFile',
+          'federator.signers.RSK.path': path.resolve(__dirname, `${keysPathResolve}reg1.key`),
+          'federator.signers.MST.type': 'keyFile',
+          'federator.signers.MST.path': path.resolve(__dirname, `${keysPathResolve}reg1.key`)
+        },
+        nodeId: '62634ab57dae9cb373a5d536e66a8c4f67468bbcfb063809bab643072d78a1243bd206c2c7a218d6ff4c9a185e71f066bd354e5267875b7683fbc70a1d455e87'
+    },
+    { // Federate node configuration, can be an array to start multiple federate nodes
+        logbackFile: customLogbackFile2,
+        classpath: classpath,
+        configFile: nodesConfigPath + 'rsk-reg-2.conf',
+        hsmConfigs: {
+          btc: {
+            serverPath: hsmV1ServerPath,
+            keyPath : keysPath + 'reg2.key'
+          },
+          rsk: {
+            serverPath: hsmV1ServerPath,
+            keyPath : keysPath + 'reg2.key',
+          },
+          mst: {
+            serverPath: hsmV1ServerPath,
+            keyPath : keysPath + 'reg2.key',
+          },
+        },
+        nodeId: 'c5946b3fbae03a654237da863c9ed534e0878657175b132b8ca630f245df04dbb0bde4f3854613b16032fb214f9cc00f75363976ee078cc4409cdc543036ccfd',
+        customConfig: {}
+    },
+    { // Federate node configuration, can be an array to start multiple federate nodes
+      logbackFile: customLogbackFile3,
+      classpath: classpath,
+       configFile: nodesConfigPath + 'rsk-reg-3.conf',
+       hsmConfigs: {
+         btc: {
+            serverPath: hsmV1ServerPath,
+            keyPath : keysPath + 'reg3.key',
+          },
+          rsk: {
+            serverPath: hsmV1ServerPath,
+            keyPath : keysPath + 'reg3.key',
+          },
+          mst: {
+            serverPath: hsmV1ServerPath,
+            keyPath : keysPath + 'reg3.key',
+          },
+       },
+       nodeId: 'cd53fc53a07f211641a677d250f6de99caf620e8e77071e811a28b3bcddf0be19e9da12b897b83765fbaebe717fab74fcb1b57c82f7978b8be3296239909e626',
+       customConfig: {}
+    }
+  ],
+  additionalFederateNodes: [
+    { // Federate node configuration, can be an array to start multiple federate nodes
+      logbackFile: customLogbackFile4,
+       classpath: classpath,
+       configFile: nodesConfigPath + 'rsk-reg-4.conf',
+       hsmConfigs: {
+        btc: {
+           serverPath: hsmV1ServerPath,
+           keyPath : keysPath + 'reg4.key',
+         },
+         rsk: {
+           serverPath: hsmV1ServerPath,
+           keyPath : keysPath + 'reg4.key',
+         },
+         mst: {
+           serverPath: hsmV1ServerPath,
+           keyPath : keysPath + 'reg4.key',
+         },
+      },
+       nodeId: '72634ab57dae9cb373a5d536e76a8c4f67468bbcfb063809bab643072d78a1243bd206c2c7a218d6ff4c9a185e71f066bd354e5267875b7683fbc70a1d455e84',
+       customConfig: {}
+    },
+    { // Federate node configuration, can be an array to start multiple federate nodes
+      logbackFile: customLogbackFile5,
+       classpath: classpath,
+       configFile: nodesConfigPath + 'rsk-reg-5.conf',
+       hsmConfigs: {
+        btc: {
+           serverPath: hsmV1ServerPath,
+           keyPath : keysPath + 'reg5.key',
+         },
+         rsk: {
+           serverPath: hsmV1ServerPath,
+           keyPath : keysPath + 'reg5.key',
+         },
+         mst: {
+           serverPath: hsmV1ServerPath,
+           keyPath : keysPath + 'reg5.key',
+         },
+      },
+       nodeId: '82634ab57dae9cb373a5d536e76a8c4f67468bbcfb063809bab643072d78a1243bd206c2c7a218d6ff4c9a185e71f066bd354e5267875b7683fbc70a1d455e85',
+       customConfig: {}
+    }
+  ]
+}

logbacks/logback-fed1.xml.sample

@@ -0,0 +1,109 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- For assistance related to logback-translator or configuration  -->
+<!-- files in general, please contact the logback user mailing list -->
+<!-- at http://www.qos.ch/mailman/listINFO/logback-user             -->
+<!--                                                                -->
+<!-- For professional support please see                            -->
+<!--    http://www.qos.ch/shop/products/professionalSupport         -->
+<!--                                                                -->
+<configuration INFO="false" scan="true" scanPeriod="3600 seconds">
+    <!-- Errors were reported during translation. -->
+    <!-- Could not find transformer for org.apache.log4j.rolling.RollingFileAppender -->
+    <appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
+        <Target>System.out</Target>
+        <encoder>
+            <pattern>%date{yyyy-MM-dd-HH:mm:ss.SSSS} %p [%c{1}]  %m%n</pattern>
+        </encoder>
+        <!-- <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
+            <level>DEBUG</level>
+        </filter> -->
+    </appender>
+    <appender name="FILE-AUDIT"
+              class="ch.qos.logback.core.rolling.RollingFileAppender">
+        <file>/PATH/TO/LOGS/fed1.log</file>
+        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
+            <Pattern>
+                %date{yyyy-MM-dd-HH:mm:ss.SSS} %p [%c{1}]  %m%n
+            </Pattern>
+        </encoder>
+        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
+            <fileNamePattern>./logs/rskj-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
+            <maxFileSize>100MB</maxFileSize>
+            <maxHistory>7</maxHistory>
+            <totalSizeCap>1GB</totalSizeCap>
+        </rollingPolicy>
+    </appender>
+
+    <logger name="execute" level="WARN"/>
+    <logger name="blockvalidator" level="WARN"/>
+    <logger name="blockexecutor" level="WARN"/>
+    <logger name="general" level="INFO"/>
+    <logger name="gaspricetracker" level="WARN"/>
+    <logger name="web3" level="DEBUG"/>
+    <logger name="repository" level="WARN"/>
+    <logger name="VM" level="WARN"/>
+    <logger name="blockqueue" level="WARN"/>
+    <logger name="io.netty" level="WARN"/>
+    <logger name="block" level="WARN"/>
+    <logger name="minerserver" level="WARN"/>
+    <logger name="minerClient" level="WARN"/>
+    <logger name="txbuilderex" level="WARN"/>
+    <logger name="pendingstate" level="WARN"/>
+    <logger name="hsqldb.db" level="WARN"/>
+    <logger name="TCK-Test" level="WARN"/>
+    <logger name="db" level="WARN"/>
+    <logger name="net" level="ERROR"/>
+    <logger name="start" level="INFO"/>
+    <logger name="co.rsk.bitcoinj.core.CheckpointManager" level="WARN"/>
+    <logger name="co.rsk.bitcoinj.wallet.Wallet" level="WARN"/>
+    <logger name="co.rsk.net.discovery.UDPServer" level="WARN"/>
+    <logger name="co.rsk.bitcoinj.core.Context" level="WARN"/>
+    <logger name="co.rsk.bitcoinj.core.PeerGroup" level="WARN"/>
+    <logger name="co.rsk.bitcoinj.core.BtcAbstractBlockChain" level="WARN"/>
+    <logger name="co.rsk.federate.bitcoin.peerdetector.BitcoinPeerDetector" level="INFO"/>
+    <logger name="co.rsk.federate.bitcoin.peerdetector.Handshaker" level="INFO"/>
+    <logger name="co.rsk.federate.FederatorSupport" level="TRACE"/>
+    <logger name="co.rsk.federate.FederateRunner" level="TRACE"/>
+    <logger name="co.rsk.federate.FedNodeRunner" level="TRACE"/>
+    <logger name="fullnoderunner" level="TRACE"/>
+    <logger name="FederationWatcher" level="WARN"/>
+    <logger name="cli" level="WARN"/>
+    <logger name="txs" level="WARN"/>
+    <logger name="gas" level="ERROR"/>
+    <logger name="main" level="INFO"/>
+    <logger name="trie" level="WARN"/>
+    <logger name="peermonitor" level="WARN"/>
+    <logger name="bridge" level="TRACE"/>
+    <logger name="org.springframework" level="WARN"/>
+    <logger name="rlp" level="WARN"/>
+    <logger name="messagehandler" level="WARN"/>
+    <logger name="sync" level="WARN"/>
+    <logger name="BtcToRskClient" level="TRACE"/>
+    <logger name="ui" level="WARN"/>
+    <logger name="java.nio" level="WARN"/>
+    <logger name="org.eclipse.jetty" level="WARN"/>
+    <logger name="wire" level="WARN"/>
+    <logger name="BridgeSupport" level="TRACE"/>
+    <logger name="jsonrpc" level="DEBUG"/>
+    <logger name="wallet" level="WARN"/>
+    <logger name="blockchain" level="WARN"/>
+    <logger name="blockprocessor" level="WARN"/>
+    <logger name="state" level="WARN"/>
+    <logger name="org.bitcoinj" level="WARN"/>
+    <logger name="metrics" level="WARN"/>
+    <logger name="messageProcess" level="WARN"/>
+    <logger name="co.rsk.pcc.NativeContract" level="TRACE"/>
+    <logger name="co.rsk.federate.bitcoin.BitcoinWrapperImpl" level="TRACE"/>
+    <logger name="co.rsk.federate.signing" level="TRACE"/>
+    <logger name="co.rsk.federate.BtcReleaseClient" level="TRACE"/>
+    <logger name="co.rsk.peg.Federation" level="TRACE"/>
+    <logger name="co.rsk.peg.RepositoryBtcBlockStoreWithCache" level="TRACE"/>
+    <logger name="co.rsk.peg.BridgeStorageProvider" level="TRACE"/>
+    <logger name="co.rsk.peg.BridgeSerializationUtils" level="TRACE"/>
+    <logger name="BridgeUtils" level="TRACE"/>
+
+    <root level="INFO">
+        <appender-ref ref="stdout"/>
+        <appender-ref ref="FILE-AUDIT"/>
+    </root>
+</configuration>