Change RKNN default core mask to all cores

[levyishai]

After some testing, we've found that using all cores all the time is the fastest option, even when running more than one object detection pipeline simultaneously. This is using RKNN_NPU_CORE_0_1_2 instead of RKNN_NPU_CORE_AUTO

[crschardt]

Can you share some testing data that supports this?

[levyishai]

Can you share some testing data that supports this?

I'm currently having trouble testing with the latest PV build (camera disconnects, camera feed looks pink for OV9782). Fwiw, that's what we've found to be the only real difference between the high-performance fork, which for us had more fps and less latency.
Hoping to find the issues with my testing setup/submit and issue/create a PR to fix the issues I'm having and bring more concrete numbers.

[levyishai]

Before updating from the fork

After changing the core mask to all cores

The current way gave me a stable 33 fps (even for two cameras), while changing it gave a stable 43 fps for one camera (which did sometimes jump to around 47), and two cameras resulted in a not-so-stable 32-37 fps.

While this might reduce efficiency for two cameras (by not so much), it ups the efficiency for the use-case which I assume is the most common - one camera. I believe it's worth a shot, or at least making this the solution when having one camera.

Feel free to test my data!

Edit: This testing was done with two OV9782 cameras at max res.

[spacey-sooty]

Can this setting be changed in the dashboard at all?

[levyishai]

Can this setting be changed in the dashboard at all?

Can be done, though this is defined on init as far as I see.

[samfreund]

Given that the performance of multiple cameras is worse, it might be desirable to have it as a toggle for teams wishing to run multiple pipelines. If you do decide to add it, I would suggest putting it under the Object Detection card in the settings.

[spacey-sooty]

I agree with Sam but would also say we need to add docs information documenting this testing so people can make an informed decision on what to select for their usecase. Happy to default on this though

[levyishai]

Given that the performance of multiple cameras is worse, it might be desirable to have it as a toggle for teams wishing to run multiple pipelines. If you do decide to add it, I would suggest putting it under the Object Detection card in the settings.

I'd love to add it as some kind of toggle button, I'm just not so sure on how to do it, especially considering this is set on init. Do you have any pointers?

[spacey-sooty]

You could either change the rknn JNI repo to allow setting it at runtime or recreate it when the option is changed

[levyishai]

You could either change the rknn JNI repo to allow setting it at runtime or recreate it when the option is changed

Just found I can use this. I'll try to see how I add something you can select from.

[levyishai]

All seems to work, would love a CR.

[spacey-sooty]

Do we have a docs section on neural networks? if so we should add that there as well

[samfreund]

yeah we do, it's about object detection

[spacey-sooty]

This behaviour should be documented there

[spacey-sooty]

Logic generally looks good but I don't have any hardware I can test this on

[steve-carpenter]

Any config change available in the interim to make this the default (w/ out PR)?

[samfreund]

Any config change available in the interim to make this the default (w/ out PR)?

You can always grab the build from this PR. Afaik, there are no config options other than that.

[spacey-sooty]

Any config change available in the interim to make this the default (w/ out PR)?

No, this PR adds a config option