-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Index updates can be significantly sped up (on some coins?) #259
Comments
can you provide what coin you experienced this with and any specific config you used with nginx? For the most part, atleast in terms of the aforementioned issues, reusing the txid wouldn't help at all as their complaints are just that there's alot of blocks, and with high number of txes per block, it can take a long time. Only real way to speed this up would be use more workers (there's only 1 worker used in sync.js) or have another way of reading the data (perhaps from a bootstrap parsing?) I'm not sure why it would be grabbing the same txid though multiple times in your case above. The logic of sync is "getblock", read the txes from the block, "getrawtransaction" per tx in the block. It shouldn't ever run into the same txid. |
Apologies, I completely forgot about this issue.
ORL, Orwell, a coin that has most likely exit scammed and hardly had any presence to begin with.
All I did was enable caching of requests to /api/ so subsequent identical(!) requests would get fetched from the Nginx cache instead of from the Iquidus service. I haven't tested Iquidus with other coins yet, I only set it up for ORL because the official explorer was gone and I needed to dive into something, using an interactive explorer is much easier than diving into RPC of course. |
Hi, can you please provide me how exactly you changed the Iquidus code to connect to Nginx? I'm facing the same slowness issue and I would like to try this solution to speed up the update, thank you |
Sadly I don't have the modified Iquidus files anymore, so I can't tell you exactly what I did, but basically all I changed was the way Iquidus used its listening port to allow Nginx to run inbetween. |
I think this is already much improved in #319, if you can test it would be great |
Merged in 8d44d4a |
I just set up Iquidus git for a small coin, and noticed the indexing process was going slower and slower.
I also noticed it was fetching the same raw transactions many times per block on the node process (see image).
It did this every ~5 or so blocks and this significantly slowed down the syncing, from several blocks per second to a block every 5-or-so seconds.
I checked, and the data it fetched was the same every time.
I messed around with it a bit and added an Nginx reverse proxy caching server inbetween, and this sped up the syncing process enormously for these entries, back up to around ~10-20 blocks per second.
If the code were to be rewritten to re-use the raw txid if it's used multiple times, this would significantly speed up the index process and would possibly even resolve/lessen issues like #104, #240 and #247.
(Edit: forgot to attach the image)
The text was updated successfully, but these errors were encountered: