Enhancements for future updates

[uaktags]

So since the #33 was closed I figured I'd start working on doing my own set of updates and was curious on what could be some things to bring in. I'm more likely not going to tackle the Plugin Support, but I have been looking at adding more customization options and APIs.

Some things I was thinking of doing:

• Update to Bootstrap 4
• Add new Bootswatch themes
• Add new APIs
• Add in Masternode support
• Cluster-ize the sync.js
• Add more exchanges
• Merge various Pull Requests

Some new APIs I'm looking for

• getStakes/:hash << Returns Stakes received by a particular address
• getStakes/:hash/since/:blockheight << Returns Stakes received by a particular address since a particular block

Some pull requests that I found to be of interest

• Get Tx detail REST API #189 which is an API addition for TX results
• Indexing for timestamp. #75 add an index to TX for performance (untested though)
• round supply to nearest coin for better display #74 Comestic change but take the advice of making it a configuration
• Make server work with IPv6 #231 Haven't tested as I only have IPv4

Any other suggestions?

[uaktags]

Some side project ideas to kind of support/compliment Iquidus:

• Perhaps https://github.com/ladimolnar/BitcoinDatabaseGenerator can be fitted for Mongo rather than MSSQL. Historically, I've personally found BDG to go through the blockchain much faster than waiting on RPC calls.

• Laravel (or any other PHP) frontend using mongodb and API calls. With Iquidus doing the heavy work of setting up the database, maybe remove the express views and just use it for API/DB generation for a PHP frontend. Then you can build out as needed via other projects.

[uaktags]

So I attempted to set the cluster based on Blocks. My initial thought process was to divide the workload (# of blocks or 'stats.count') by the 4 (small test to check how well it scales). This required changes to sync.js. It was promising in the beginning as the coin I tested had 10k blocks that only had a single tx, so in a matter of 7minutes we had all but 1 worker finished working. This meant that the remaining blocks, some of which had 1k txes each) were still left to a single process. We completed our test at only shaving 1-2 minutes which isn't anything.

So I then I started to think that perhaps we could do it on a basis of splitting the txes per block to each worker. So foreach blocks (stats.count) -> foreach transaction (block[i].tx.length) assign tx to a worker up to 4 workers. However, with this method, I've yet to figure out the logic and code needed to create/run this test.

So yea....roadblock on clusterizing the sync.

[uaktags]

Themes added/removed:
Updating to BS4 required some changes and meant that themes that bootswatch removed needed to be removed as well.

Removed:

• Readable
• Paper

Added:

• Litera
• Lux
• Minty
• Materia
• Pulse
• Sketchy
• Solar

[uaktags]

I've started working on Security Vulnerabilities as presented by npm audit.
NPM audit fix, takes care of the bulk, but bitcoin-node-api hasn't been updated in a while, and forks that I've seen like chainnode still use the same outdated deps.
So I've replaced bitcoin with bitcoin-core, forked and updated the bitcoin-node-api to use bitcoin-core, and updated its deps for the express versions. Then went ahead and updated iquidus for all it needs, even replacing Jade with Pug.

Now is the tedious task of fixing anything that was removed/changed between Jade and Pug2 migration in the jade files.

[uaktags]

I implemented #256 as it definitely does feel faster. Only downside is the console is filled with urlencode junk from the table refresh.

[uaktags]

Started incorporating @suprnurd 's ciquidus bitcoin-node-api functions. However, since the crypto-world is opensourced, there's no definitive schema for commands. Vulcano, which I test on, has masternode functions that differ from chaincoins. As long as there's different coins, and different coins can have different apis, then we need to be able to dynamically load them without changing dependencies.

My solutions was to create a folder to store json files.

./coin_commands

./coin_commands/base.json
{
   "heavy":[
      "getinfo",
      "getstakinginfo",
      "getnetworkhashps",
      "getdifficulty",
      "getconnectioncount",
      "getblockcount",
      "getblockhash",
      "getblock",
      "getrawtransaction",
      "getmaxmoney",
      "getvote",
      "getmaxvote",
      "getphase",
      "getreward",
      "getnextrewardestimate",
      "getnextrewardwhenstr",
      "getnextrewardwhensec",
      "getsupply",
      "gettxoutsetinfo"
   ],
   "default":[
      "getinfo",
      "getnetworkhashps",
      "getmininginfo",
      "getdifficulty",
      "getconnectioncount",
      "getblockcount",
      "getblockhash",
      "getblock",
      "getrawtransaction",
      "getpeerinfo",
      "gettxoutsetinfo"
   ]
}

./coin_commands/vulc.json
{
	"heavy":[
	   "getmasternodecount",
	   "getmasternodeoutputs",
	   "getmasternodescores",
	   "getmasternodestatus",
	   "getmasternodewinners",
	   "listmasternodes",
	   "masternode",
	   "masternodecurrent"
   ],
   "default":[
		"getmasternodecount",
		"listmasternodes"
   ]
   
}

Then, based off of my settings.json file, I decide which commands are needed:

 "commands_needed": [
    "base.json", "vulc.json"
  ],

Then during the app.js load, it now does the following logic:

var commands = [];
    for(i=0; i < settings.commands_needed.length; i++){
      var cmds = JSON.parse(fs.readFileSync("coin_commands/"+settings.commands_needed[i]));
      var cmd = (settings.heavy ? cmds.heavy : cmds.default);
      for(k=0; k < cmd.length; k++){
        commands.push(cmd[k]);
      }
    }
  bitcoinapi.setAccess('only', commands);

Now we can modularly support any coin without changing the code. Just specify what coin commands you need. This could use some more thought/work, but it works for a proof of concept.

[uaktags]

Okay, started working on getting social settings a bit more streamlined. Currently, if you need to add/remove a link (website, google+, etc) you'll add it to settings.js, settings.json, and then add it to the template. I've instead decided to add a "social" object which then gets parsed and templated dynamically.

settings.json

"social":{
    "twitter": {
      "url":"https://twitter.com/iquidus",
      "enabled": false,
      "fontawesome": "fa-twitter"
    },
    "facebook": {
      "url":"yourfacebookpage",
      "enabled": false,
      "fontawesome": "fa-facebook"
    },
    "googleplus": {
      "url":"yourgooglepluspage",
      "enabled": false,
      "fontawesome": "fa-google-plus"
    },
    "youtube": {
      "url":"youryoutubechannel",
      "enabled": false,
      "fontawesome": "fa-youtube"
    },
    "bitcointalk": {
      "url":"yourbitcointalktopicvalue",
      "enabled": false,
      "fontawesome": "fa-btc"
    },
    "github": {
      "url":"yourgithubusername/yourgithubrepo",
      "enabled": false,
      "fontawesome": "fa-github"
    },
    "slack": {
      "url":"yourfullslackinviteurl",
      "enabled": false,
      "fontawesome": "fa-slack"
    },
    "website": {
      "url":"yourfullwebsiteurl",
      "enabled": false,
      "fontawesome": "fa-link"
    }
  },

app.js

app.set("social", settings.social);

layout.jade

.col-md-4
        ui.nav.justify-content-left
          each social in settings.social
            if social.enabled
                li.pull-left(style="margin-right:1%")
                  a#website-icon(href=social.url)
                    span(class="glyphicon fa "+ social.fontawesome)

Now if I wanted to, I could completely delete out twitter,slack, etc all the things I don't want. I can then add in discord

"discord": {
      "url":"myDiscordJoinLink",
      "enabled": true,
      "fontawesome": "fa-discord"
    },

No longer does anything need to be recoded, it's a setting, just add/remove your settings.

[uaktags]

With #257 and commit bc25fe0 I started work on the api for verifymessage. This isn't built into the BitcoinNodeAPI so I had to start doing that as well. Basically added verifymessage as a SpecialAPICase, urlDecoded the signature, and I was good to go. VerifyMessage only returns True/False and then the help/man output for Verifymessage, so some work is needed for parsing that. Later I'll add labels to the database (or maybe just add a new field to addresses).

Basic premise is to have a form on /address/:hash/claim. ":hash" is a disabled input, and the user is presented with a field for a "display name" and "signature". These correspond to "message" and "signature" respectively. They validate they can sign a message, they own that address, they can set the name.

Pretty simple concept.

[uaktags]

this has been done and appears complete.
Claimed Address

Unclaimed Address

Claim Form

[uaktags]

So if you're following everything in uaktags/explorer or in my PR, then you'll know that I've gone ahead and cluster-ized the sync process. However, once you do, things get a bit unstable. I'll be pushing it as a settings option for others to try it as they wish.

[BlockMechanic]

Wonderful work on this !!! I'll try to remember to setup donation addresses for you in every coin i help out and use your fork.