Skip to content

Dtananaev/lidar_dynamic_objects_detection

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

50 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Dynamic objects detection in LiDAR

MIT License

The result of network (click on the image below)

result The network weights could be loaded weight.

Installation

For ubuntu 18.04 install necessary dependecies:

sudo apt update
sudo apt install python3-dev python3-pip python3-venv

Create virtual environment and activate it:

python3 -m venv --system-site-packages ./venv
source ./venv/bin/activate

Upgrade pip tools:

pip install --upgrade pip

Install tensorflow 2.0 (for more details check the tensofrolow install tutorial: tensorflow)

pip install --upgrade tensorflow-gpu

Clone this repository and then install it:

cd lidar_dynamic_objects_detection
pip install -r requirements.txt
pip install -e .

This should install all the necessary packages to your environment.

The method

The lidar point cloud represented as top view image where each pixel of the image corresponds to 12.5x12.5 cm. For each grid cell we project random point and get the height and intensity

We are doing direct regression of the 3D boxes, thus for each pixel of the image we regress confidence between 0 and 1, 7 parameters for box (dx_centroid, dy_centroid, z_centroid, width, height, dx_front, dy_front) and classes.

We apply binary cross entrophy for confidence loss, l1 loss for all box parameters regression and softmax loss for classes prediction. The confidence map computed from ground truth boxes. We assign the closest to the box centroid cell as confidence 1.0 (green on the image above) and 0 otherwise. We apply confidence loss for all the pixels. Other losses applied only for those pixels where we have confidence ground truth 1.0.

The dataset preparation

We work with Pandaset dataset which can be uploaded from here: Pandaset Upload and unpack all the data to dataset folder (e.g. ~/dataset). The dataset should have the next folder structure:

    dataset
    ├── 001                     # The sequence number
    │   ├── annotations         # Bounding boxes and semseg annotations
    |   |   ├──cuboids
    |   |   |  ├──00.pkl.gz
    |   |   |  └──  ...
    |   |   ├──semseg
    |   |      ├──00.pkl.gz
    |   |      └── ...
    │   ├── camera             # cameras images
    |   |  ├──back_camera
    |   |  |  ├──00.jpg
    |   |  |  └── ..
    |   |  ├──front_camera
    |   |  └── ...
    │   ├── lidar             # lidar data|    ├── 00.pkl.gz
    │   |    └── ... 
    |   ├── meta
    |   |   ├── gps.json
    |   |   ├── timestamps.json
    ├── 002
    └── ...

Preprocess dataset by applying next command:

cd lidar_dynamic_objects_detection/detection_3d/data_preprocessing/pandaset_tools
python preprocess_data.py --dataset_dir <path_to_your_dataset_dir>

Create dataset lists:

cd lidar_dynamic_objects_detection/detection_3d/
python create_dataset_lists.py --dataset_dir <path_to_your_dataset_dir>

This should create train.datatxt and val.datatxt into your dataset folder. Finally change into parameters.py the directory of the dataset.

Train

In order to train the network:

python train.py

In order to resume training:

python train.py --resume

The training can be monitored in tensorboard:

tensorboard --logdir=log

Inference on validation dataset

In order to do inference on validation dataset:

python validation_inference.py --dataset_file <path_to_dataset_folder>/val.datatxt --output_dir <path_to_inference_output> --model_dir <path_to_trained_model>

The result of the inference is 3d boxes and also visualized 3d boxes on top view image. The visualized top view image (upper) concatenated with ground truth top view image (bottom).

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages