Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Theoretical question regarding global vs rolling shutter behaviour and accounting for it with ZED2 and D455 #464

Open
Aishkrish18 opened this issue Aug 15, 2024 · 2 comments

Comments

@Aishkrish18
Copy link

Aishkrish18 commented Aug 15, 2024

I have a question. Couple of months ago, I had performed detailed tests between ZED2 stereo inertial mode and D455 with OpenVINS , resulting in good performance with ZED2 camera. But from some of the issues, i see that it is a rolling shutter camera, and the algorithm doesn't support rolling shutter. But D455 on the other hand gave relatively poor performance with bigger offset compared to ZED2 although it is a global shutter camera.
This is with ZED2
zed2_openvins drawio

This is with D455
openvins_d455 drawio
Both of them are compared in reference to motion capture system.
May i get some insights on what could be the reason for this behaviour?

@Aishkrish18
Copy link
Author

Aishkrish18 commented Aug 16, 2024

to put it better, how does zed2 perform so well although its a rolling shutter camera, I'm unable to reason out this part

@Userpc1010
Copy link

Why are you sure that the camera is the reason? The VINS system is primarily an inertial navigation system corrected by visual odometry, i.e. INS serves as a kind of battery analog for smoothing out visual odometry errors. So, I don't know about zed cameras, but the d455 uses the Bosch BMI 055 IMU and it is a bad and inaccurate sensor, and the synchronization of the camera and IMU leaves much to be desired. In datasets with stereo cameras, imu synchronization is used based on the average value of the camera shutter, i.e. they make an interruption at the start of the camera shutter charge and then find the average time value at the end of the charge, and this value is closer and more accurate than in the d455 (at least it was before) a time stamp is used only from the start of the camera shutter charge. But the exposure time is not a constant, it changes depending on the strength of the ambient light, therefore the delay between the real movement in the frame and the one received from the imu also changes, which leads to errors until openvins adjusts to the new delays (lighting conditions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants