FlashMix: Fast Map-Free LiDAR Localization via Feature Mixing and Contrastive-Constrained Accelerated Training
📖 Paper (To appear): WACV 2025
📖 Pre-print: arXiv
📹 Video: Youtube
Control/Robotics Research Laboratory (CRRL), Department of Electrical and Computer Engineering, NYU Tandon School of Engineering
-
Map-free LiDAR localization framework combining a pre-trained point encoder and a scene-specific pose regressor, with feature buffer enabled rapid training.
-
MLP-Mixer based descriptor aggregator to fuse global point relationships by feature mixing for adapting to scene-specific geometries.
-
Contrastive and metric learning loss based regularization to enhance global descriptor retrieval performance and stable convergence while maintaining fast training times.
-
Extensive experiments in outdoor and indoor environments, demonstrating rapid training and adaptation with competitive performance compared to existing map-free localization methods.
Fig. 1: Comparision of LiDAR pose regression-based framework (top) with our fast map-free LiDAR localization system.b
conda create --name flashmix python=3.10.11
conda activate flashmix
pip install numpy pyyaml opencv-python matplotlib
pip install torch --index-url https://download.pytorch.org/whl/cu121
pip install torch-scatter -f https://data.pyg.org/whl/torch-2.4.0+cu121.html
pip install torch-sparse -f https://data.pyg.org/whl/torch-2.4.0+cu121.html
pip install torch-cluster -f https://data.pyg.org/whl/torch-2.4.0+cu121.html
pip install -r requirements.txtInstall SpTr from source.
We support the Oxford Radar RobotCar, vReLoc, and Mulran DCC datasets right now.
The data should be organized as follows:
data_root
├-Oxford-Radar
| ├-2019-01-11-14-02-26-radar-oxford-10k
| ├-velodyne_left_False.h5
| ├-velodyne_left
| ├-2019-01-14-12-05-52-radar-oxford-10k
| ├-2019-01-14-14-48-55-radar-oxford-10k
| ├-2019-01-15-13-06-37-radar-oxford-10k
| ├-2019-01-17-13-26-39-radar-oxford-10k
| ├-2019-01-17-14-03-00-radar-oxford-10k
| ├-2019-01-18-14-14-42-radar-oxford-10k
| ├-2019-01-18-15-20-12-radar-oxford-10k
├-vReLoc
| ├-seq-03
| ├-seq-05
| ├-seq-06
| ├-seq-07
| ├-seq-12
| ├-seq-14
| ├-seq-15
| ├-seq-16
├-Mulran/DCC
| ├-DCC1
| ├-DCC2
| ├-DCC3
velodyne_left_False.h5 files can be generated using the data generator in PosePN. Alternately, we provide the pre-generated file here.
cd src/utils/
python create_robotcar_pickle.py --data_dir <path_to_Oxford-Radar> --save_dir <path_to_save_pickle>
python create_dcc_pickle.py --data_dir <path_to_Oxford-Radar> --save_dir <path_to_save_pickle>
python create_vreloc_pickle.py --data_dir <path_to_Oxford-Radar> --save_dir <path_to_save_pickle>
Navigate to the base, create a folder inside src named checkpoints to save the trained models.
mkdir -p src/checkpoints/FlashMix src/checkpoints/SALSA/Model
Download pre-trained weights for the backbone (feat_extractor.pth) of SALSA from this link and save in src/checkpoints/SALSA/Model.
python src/main.py --dataset <robotcar or dcc or vReLoc>
The default config files for each dataset are located at src/config.
The trained models will be saved in the src/checkpoints/FlashMix directory.
python src/eval.py --dataset <robotcar or dcc or vReLoc>
Our pre-trained models can also be downloaded from this link. After downloading, copy the contents into the src/checkpoints/FlashMix directory.
The relocalization rates of the best-performing models as a function of training time are plotted below.
Fig. 2: Analysis of relocalization rate as a function of train time
Fig. 3a: Oxford-Radar
Fig. 3b: Mulran DCC
Fig. 3c: vReLoc
Fig. 3: Visualization of different methods on test trajectories from Oxford-Radar, DCC, and vReLoC datasets. Trajectory visualization: The ground truth and estimated positions are shown in dark blue and red dots, respectively. The star shows the starting position.b
If you find our work useful in your research please consider citing our publication: