Suggestions for Resolving Frame Drop Issues and hardware sync with RealSense Camera at 60Hz

[Akumar201]

I am experiencing frame drop issues while using a RealSense 405 camera at 60Hz. My primary requirement is to capture data consistently at 60Hz or higher without dropped frames. I understand that multiple factors can affect performance, including the camera model, cable quality, and system configuration. However, I wanted to seek guidance from experts who may have tested and validated solutions for this issue. Specifically, I would appreciate recommendations on:

1. The best RealSense camera model for achieving stable 60Hz performance.
2. Any specific hardware or cables that could help minimize frame drops.
3. Whether hardware synchronization (Intel's hardware sync guide) can help in this scenario. From my understanding, hardware sync may have limitations in terms of achievable frame rates, but I would love any insights on this.

Ideally it would be great if a camera can do 60HZ with no frame drop and hardware sync option. Such that I can attach multiple camera but get the data synchronized to my computer.

Thank You

[MartyG-RealSense]

Hi @Akumar201 The D405 camera model is not equipped with a set of connector pins necessary for hardware sync that other RealSense 400 Series camera models have.

Hardware sync is not likely to be of benefit for the goal of reducing frame drops, but is very helpful if you will be using multiple cameras and want to try to sync them to the timing of a master signal.

If a USB cable longer than 2 meters is used then as you mentioned, cable quality will be important . It should also be a cable that is designed for data transfer and not an inexpensive charging / device sync cable.

For 5 meters length or longer, a cable with 'active repeater' components inside it to boost the signal is recommendable.

A programming measure that can reduce the risk of frame drops is to add latency by increasing the frame queue size so that the pipeline can hold a greater number of frames simultaneously (by default, the pipeline can only hold 16 frames from each enabled stream type, with old frames dropping out as new ones arrive). This will increase the amount of computer memory consumed though, so a computer with good memory capacity will be beneficial when using this technique.

https://github.com/IntelRealSense/librealsense/wiki/Frame-Buffering-Management-in-RealSense-SDK-2.0#latency-vs-performance

The instruction Keep() can also be helpful for performance, as it allows you to store frames in the computer's memory instead of continually writing them to the computer's storage device and then perform a mass batch-processing action on the frames all at the same time when the pipeline is stopped (alignment, post-processing, save to file, etc). Because Keep() progressively consumes memory as more frames are stored though, you can only store about 30 seconds worth of frames on a PC with a good memory capacity unless you periodically release some of the frames from memory.

If you are writing your own program script that will use multiple cameras then you should use the instruction poll_for_frames() instead of wait_for_frames(), as described at #2422 (comment)

Aside from D405 which lacks hardware sync pins, any 400 Series camera model (D415, D43x, D45x) should be suitable and perform similarly in your multicam hardware-synced setup when streaming depth at 60 Hz. The maximum resolution that you can use at 60 Hz though will be 848x480 as 1280x720 only supports 30 maximum.