Problem Initializing IMU for OrbSlam3 on EuRoC dataset

[migsdigs]

Hi there,

Thank you for work and for providing such a useful repo. I was hoping you could help me with an issue I am having when trying to run Orbslam in mono-inertial mode with IMU and image data from the EuRoC data.

I see it is quite a common issue from some of the already existing issues posted, but the recommendations I tried did not fix my issue. Like with others, I get this message in a loop when trying to run the slam framework with images and imu data:

First KF:82; Map init KF:71
New Map created with 418 points
Point distribution in KeyFrame: left-> 418 --- right-> 0
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 389
mnInitialFrameId = 387
332 Frames set to lost
First KF:84; Map init KF:82
New Map created with 531 points
Point distribution in KeyFrame: left-> 531 --- right-> 0
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 393
mnInitialFrameId = 391
337 Frames set to lost
First KF:86; Map init KF:84
New Map created with 237 points
Point distribution in KeyFrame: left-> 237 --- right-> 0
build optimization graph
start optimization
end optimization
update Keyframes velocities and biases
IMU in Map 0 is initialized
PIOLKF: NOT ENOUGH EDGES
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
FAIL LOCAL-INERTIAL BA!!!!
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 401
mnInitialFrameId = 397
382 Frames set to lost

I have the following parameters set in my yaml settings file:

%YAML:1.0

#--------------------------------------------------------------------------------------------
# Camera Parameters. Adjust them!
#--------------------------------------------------------------------------------------------
File.version: "1.0"

Camera.type: "PinHole"

# Camera calibration and distortion parameters (OpenCV)
Camera.fx: 458.654
Camera.fy: 457.296
Camera.cx: 367.215
Camera.cy: 248.375

Camera.k1: -0.28340811
Camera.k2: 0.07395907
Camera.p1: 0.00019359
Camera.p2: 1.76187114e-05

# Camera resolution
Camera.width: 752
Camera.height: 480

#Camera.newWidth: 600
#Camera.newHeight: 350

# Camera frames per second
Camera.fps: 20

# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1

# Transformation from camera to body-frame (imu)
Tbc: !!opencv-matrix
   rows: 4
   cols: 4
   dt: f
   data: [0.0148655429818, -0.999880929698, 0.00414029679422, -0.0216401454975,
          0.999557249008, 0.0149672133247, 0.025715529948, -0.064676986768,
          -0.0257744366974, 0.00375618835797, 0.999660727178, 0.00981073058949,
          0.0, 0.0, 0.0, 1.0]


IMU.NoiseGyro: 1.6968e-04
IMU.NoiseAcc: 2.0e-3
IMU.GyroWalk: 1.9393e-05
IMU.AccWalk: 3.0000e-03 # 3e-03
IMU.Frequency: 200

#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------

# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 2000 # 1000

# ORB Extractor: Scale factor between levels in the scale pyramid
ORBextractor.scaleFactor: 1.2

# ORB Extractor: Number of levels in the scale pyramid
ORBextractor.nLevels: 8

# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7

#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1.0
Viewer.GraphLineWidth: 0.9
Viewer.PointSize: 2.0
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3.0
Viewer.ViewpointX: 0.0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -3.5 # -1.8
Viewer.ViewpointF: 500.0

I understand that the common issue can be that the transformation matrix from camera frame to body (IMU) is incorrect. However, I have got this directly from the calibration/sensor configuration files provided in the EuRoC dataset and they also appear to be the same as those provided in the yaml file for your EuRoC example. Furthermore, I would imagine that in a dataset like EuRoC there is sufficient excitation along all axes.

If you have any other recommendations, I would greatly appreciate them.

For further context, I have been using a Python wrapper for OrbSlam, and it has been able to work just fine for the regular monocular case without IMU data, on KITTI data and my own.

Thank you for your time.