Discrepancies in RMS ATE results from Kimera-VIO paper

[vasMil]

Hello,

I have been trying to replicate the results from your paper, but I am encountering some discrepancies in the RMS ATE values I get when compared to the ones reported. Additionally, I noticed that the ORB-SLAM3 paper also reports different values for your work, which adds to my confusion.

Here are the steps I followed:

1. Cloned the repository and built the Docker image.
2. Downloaded the dataset files provided in the Google Drive.
3. Ran the Kimera-VIO script for each dataset.
4. Converted traj_vio and traj_gt files into TUM format.
5. Used the evo tool to calculate RMS ATE. For this, the trajectories were aligned using the provided Umeyama technique and finally the RMSE was calculated, using the APE.
   • The guide I followed: evo-notebook

In the table below you can quickly compare the differences. A colleague also attempted to replicate these results and faced similar issues.

I was wondering if I might have missed something in my process and whether the differences I am seeing are expected.

Dataset	Paper RMSE results for (fixed-lag smoothing)	Our RMSE results
MH_1	0.11	0.23
MH_2	0.10	0.20
MH_3	0.16	0.37
MH_4	0.24	0.29
MH_5	0.35	0.26
V1_1	0.05	0.06
V1_2	0.08	0.06
V1_3	0.07	0.09
V2_1	0.08	0.09
V2_2	0.10	0.15
V2_3	0.21	0.27

Thank you!

[alicaank]

Hi @vasMil could you please help me about run Kimera-VIO on TUM RGBD dataset?