-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement/use a byte slice pool #254
base: master
Are you sure you want to change the base?
Conversation
Adds a new pool for bytes to help with gc slowdowns. Common package is needed to avoid import cycles when putting freeBytes into util package
f43aba3
to
6bdb889
Compare
There are functions in these packages that allocate and de-allocate byte slices numerous times per function call. Using a pool for byte slices help reduce this allocation and speeds up the code.
a7ed94d
to
4d73eb7
Compare
Hey - thanks for working on this but as we talked about on the call, I'm pretty hesitant to use this. It's not actually that complex, but it is kind of weird, and people coming to the code for the first time would be like "why is it doing this weird thing". Or people working on code / modifying it later may not use it as it's not obvious that they need to. It seems like most of the code that gets sped-up by this is just dumb code that we should actually fix.
That probably eliminates the extra allocation. Though we can probably do even better... (I still think some kind of hash worker goroutine that then gets The other problems mentioned were Similarly leaf hashes don't track their own hash, but they could, just like btcutil txs do. So it makes sense that a slice pool will help because there's a bunch of dumb / slow code here, but I think the first thing we should do is fix the dumb / slow code, which might end up faster than the pool. Once we fix these things, if it turns out the pool still makes a significant help, then I'd be OK using one. If there are other slow / CG or allocation heavy areas, please let me know / write them here and I'll take a look. |
Hm yeah just thinking... even without threads, keeping everything blocking, the ParentHash function could become a method on forest or pollard, and both those structs just have their own Also there's all sorts of hardware optimized hashing that I don't think we're using right now. |
There are functions in these packages that allocate and de-allocate byte
slices numerous times per function call. Using a pool for byte slices
help reduce this allocation and speeds up the code.
Also adds helper functions for serializing and de-serializing ints to byte slices for the byte slices returned from the pool as functions such as
binary.BigEndian.PutUint64
binary.BigEndian.Uint64
allocate bytes inside the functions.Also added comments for types/functions that didn't have any