-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: parallelize registry builds #86
Conversation
parallelize calls to nodes in order to speed up building of the ATA registry fix issue where existing poolAssets, foreignAsssets and assets Info were not being preserved when updating the registry
update fetChainInfo inline doc
update logged message relay chain message
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
I think this is a great first step to parallelizing the calls. But we should continue to increase the performance by isolating the connections into parallel connections. So currently we just add each chains process as a single clump into a Promise.all, but if we want to make it even faster we can take each rpc connection under a single chain and run them all at the same time.
Here's how we do it in sidecar for /blocks?range=
: https://github.com/paritytech/substrate-api-sidecar/blob/master/src/util/PromiseQueue.ts
We can also use generators in JS to achieve this but the above Promise queue is a little faster. We can also set a max size so we don't run to many connection at parallel.
For example: Under Kusama: we can take every parachain, and run all the rpc connections, and calls in clumps of 5-10. and so on
Let's merge this in as its a big upgrade, then we can focus on increasing performance even more. It might take a bit of refactoring to get each rpc connection isolated into a Promise Queue. |
XcmV3MultiLocation
type construction and replaces with custom XCM MultiLocation typecloses: #81