Using multiple cores while searching #154
Replies: 2 comments 4 replies
-
|
For single point search this is, in practice, very hard. It may be possible to thread over batches of queries however. That had significant impacts on single query performance originally, but there may but there may be workarounds. I'll look into it. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I've meant batches of queries. For example, my current task uses 'train' dataset with ~10 million vectors with a length of 30, and 'test' dataset with ~5 million vectors. In single core mode it is really slow to call for 5 million KNN searches. In such cases, a single query performance is the last thing to care about, even if it will be 10 times slower than now. It will be great if you would ever consider to implement batch multithreading. Thank you again for the time you've spend for making this recent documentation improvement! |
Beta Was this translation helpful? Give feedback.
-
|
Streaming queries are also a common use case, and that tends to require lots of single queries to be fast. This is, for example, what ann-benchmarks tests. I took a look at things, however, and given how numba handles things, and with a little fine tuning, I think this can be done relatively easily. See the branch |
Beta Was this translation helpful? Give feedback.
-
|
I somehow failed to install this via setup.py, so I've just replaced pynndescent_.py with a new version and removed the pycache folder. But I don't think this could be a reason for the errors that occured. The index is creating without errors, but when I try to pickle.dump it, this error message appears: The same error is shown when I skip pickle and try to query neighbors: In master version of pynndescent everything goes fine. My installed pynndescent version is 0.5.5, numba version is 0.54.1 |
Beta Was this translation helpful? Give feedback.
-
|
I suspect something went wrong with the replacement/install somehow, because this should not really be possible as it stands, and there is no difference with the main branch with respect to how any of that part of the code should work. None the less I've pushed a change that should resolve that particular issue by force to the branch. Let me know if that fixes it. |
Beta Was this translation helpful? Give feedback.
-
|
This time I've got more luck in installation, so unfortunately, I can't tell either your fix worked or it was an installation problem. Anyway, batch multithreading works fine, here are some timings: |
Beta Was this translation helpful? Give feedback.
-
Hello and thanks for the great job you've done! As far as I could see, there is no way to use multiple cores while searching nearest neighbors. Are there any plans to add this feature in the future? Thank you in advance!
Beta Was this translation helpful? Give feedback.
All reactions