Camvox

tiagodts · Nov 19, 2020 · 81b7694 · 81b7694
1 parent 219dee5
commit 81b7694
Show file tree

Hide file tree

Showing 724 changed files with 56,805 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -1,2 +1,126 @@
 # CamVox
-A low-cost SLAM system based on camera and Livox lidar.
+## A Low-cost and Accurate Lidar-assisted Visual SLAM System
+
+<img src="./pics/CamVox.gif" alt="show" />
+
+We propose **CamVox** by adapting Livox lidars into visual SLAM (ORB-SLAM2) by exploring the lidars’ unique features. Based on the non-repeating nature of Livox lidars, we propose an automatic lidar-camera calibration method that will work in uncontrolled scenes. The long depth detection range also beneﬁt a more efﬁcient mapping. Comparison of CamVox with visual SLAM (VINS-mono) and lidar SLAM (livox_horizon_loam) are evaluated on the same dataset to demonstrate the performance.
+
+<div align="center">
+    <img src="pics/compare.png" width = 100% >
+</div>
+
+**Developer:** [Yuewen Zhu](https://github.com/zywok), [Chunran Zheng](https://github.com/xuankuzcr), [Chongjian Yuan](https://github.com/ChongjianYUAN)
+
+**Our related video**: our related videos are now available on [[YouTube Video](https://www.youtube.com/watch?v=AUnZNBB-uUE)] [[bilibili Video](https://www.bilibili.com/video/BV1fZ4y1V795/)]. 
+
+## 1. Prerequisites
+### 1.1 Ubuntu and ROS
+Ubuntu 64-bit 16.04 or 18.04.
+ROS Kinetic or Melodic. Follow [ROS Installation](http://wiki.ros.org/ROS/Installation).
+
+### 1.2 Pangolin
+We use [Pangolin](https://github.com/stevenlovegrove/Pangolin) for visualization and user interface. Follow [Pangolin Installation](https://github.com/stevenlovegrove/Pangolin).
+
+### 1.3 OpenCV
+We use OpenCV to manipulate images and features. Follow [Opencv Installation](http://opencv.org). **Required at leat 2.4.3. Tested with OpenCV 2.4.11 and OpenCV 3.2**.
+
+### 1.4 Eigen3
+Follow [Eigen Installation](http://eigen.tuxfamily.org). **Required at least 3.1.0**.
+
+### 1.5 Ceres Solver
+Follow [Ceres Installation](http://ceres-solver.org/installation.html).
+
+## 2. Build CamVox
+Clone the repository and catkin_make:
+
+```bash
+    cd ~/catkin_ws/src
+    git clone https://github.com/ISEE-Technology/CamVox && cd ..
+    catkin_make
+	cd src/CamVox/isee-camvox && chmod +x build_camvox.sh	
+	./build_camvox.sh
+    source ~/catkin_ws/devel/setup.bash
+```
+## 3. Run with Hardware
+### 3.1 Hardware
+
+ <table>
+	<tr>
+	    <th>Platform</th>
+	    <th>Item</th>
+	    <th>Pics</th>  
+	    <th>Shopping Link</th> 
+	</tr >
+	<tr >
+	    <td rowspan="5"><img src="./pics/platform.png" /></td>
+	    <td>Livox Horizon </td>
+	    <td align="center" valign="middle"><img src=  "./pics/horizon.jpg" width=25% /></td>
+            <td align="center" valign="middle">  <a href ="https://www.livoxtech.com/horizon"> Lidar </a> </td>
+	</tr>
+	<tr>
+	    <td> MV-CE060-10UC</td>
+	    <td align="center" valign="middle"><img src="./pics/camera.png" width=19% /></td>
+	    <td align="center" valign="middle">  <a href ="https://en.hikrobotics.com/vision/visioninfo.htm?type=42&oid=2451"> Camera </a> </td>
+	</tr>
+	<tr>
+	    <td> Inertial Sense uINS </td>
+	    <td align="center" valign="middle"> <img src="./pics/Inertial_Sense_uINS.png" width=22% /> </td>
+	    <td align="center" valign="middle"> <a href ="https://inertialsense.com/product/rugged-µins/"> RTK </a> </td>
+	</tr>
+	<tr>
+	    <td>Manifold2C</td>
+	    <td align="center" valign="middle"><img src="./pics/Manifold2C.jpg" width=22% /></td>
+            <td align="center" valign="middle">  <a href ="https://www.dji.com/cn/manifold-2"> Onboard-Computer </a> </td>
+	</tr>
+	<tr>
+	    <td> Scout-mini </td>
+	    <td align="center" valign="middle"><img src="./pics/Scout-mini.jpg" width=28% /></td>
+	    <td align="center" valign="middle">  <a href ="http://www.agilex.ai/index/product/id/3?lang=zh-cn"> Robot Chassis </a> </td>
+	</tr>
+</table>
+
+### 3.2 Hard Synchronization
+
+Hard synchronization is performed with all of these sensors by a trigger signal of 10 Hz. The camera output at each trigger signal(10 Hz). The lidar keeps a clock (synced with GPS-RTK) and continuously outputs the scanned point with an accurate timestamp. In the meantime, the IMU outputs at a frequency of 200 Hz synced with the trigger. The Hardware Synchronization diagram is as follows.
+
+<img src="./pics/synchronization.jpg" width=100% />
+
+### 3.3 Running
+
+Connect to your PC to Livox Horizon lidar by following  [Livox-ros-driver installation](https://github.com/Livox-SDK/livox_ros_driver).
+
+```bash
+    chmod +x run.sh
+	./run.sh
+```
+
+
+## 4. Run with Rosbag Example
+
+### 4.1 SUSTech Dataset (Loop Closure)
+
+We open sourced our dataset in SUSTech campus with loop closure. [Download here](https://drive.google.com/file/).
+
+### 4.2 Rosbag Example with static scenes (Automatic Calibration trigger)
+
+We provide a rosbag file with static scenes to test the automatic calibration thread. [Download here](https://drive.google.com/file/). <br/>
+When the car detects more than **20 frames of still images (about 2 seconds)**, the automatic calibration thread starts to work. The thread will be interrupted to enter the SLAM mode if the car starts to move before the end of calibration. 
+The effects of automatic calibration is shown as follows.
+
+<img src="./pics/runyang.gif" alt="show" width = 49% /> <img src="./pics/flat.gif" alt="show" width = 49%  />
+
+### 4.3 Running
+```bash
+    chmod +x run.sh
+	./run.sh
+    rosbag play YOUR_DOWNLOADED.bag
+```
+
+## 5. Acknowledgements
+The authors thank colleagues at Livox Technology for helpful discussion and support. <br/>
+The repository is from [**ISEE**](https://isee.technology/).
+
+## 6. License
+
+The source code is released under [GPLv2](http://www.gnu.org/licenses/) license.
+
diff --git a/inertial-sense-ros b/inertial-sense-ros
diff --git a/isee-camvox b/isee-camvox
diff --git a/isee-preprocessing/.vscode/c_cpp_properties.json b/isee-preprocessing/.vscode/c_cpp_properties.json
@@ -0,0 +1,17 @@
+{
+    "configurations": [
+        {
+            "name": "Linux",
+            "includePath": [
+                "${workspaceFolder}/**"
+            ],
+            "defines": [],
+            "compilerPath": "/usr/bin/gcc",
+            "cStandard": "gnu11",
+            "cppStandard": "c++17",
+            "intelliSenseMode": "gcc-x64",
+            "compileCommands": "${workspaceFolder}/build/compile_commands.json"
+        }
+    ],
+    "version": 4
+}
diff --git a/isee-preprocessing/.vscode/settings.json b/isee-preprocessing/.vscode/settings.json
@@ -0,0 +1,65 @@
+{
+    "files.associations": {
+        "ostream": "cpp",
+        "cctype": "cpp",
+        "clocale": "cpp",
+        "cmath": "cpp",
+        "csignal": "cpp",
+        "cstdarg": "cpp",
+        "cstddef": "cpp",
+        "cstdio": "cpp",
+        "cstdlib": "cpp",
+        "cstring": "cpp",
+        "ctime": "cpp",
+        "cwchar": "cpp",
+        "cwctype": "cpp",
+        "array": "cpp",
+        "atomic": "cpp",
+        "strstream": "cpp",
+        "*.tcc": "cpp",
+        "bitset": "cpp",
+        "chrono": "cpp",
+        "complex": "cpp",
+        "cstdint": "cpp",
+        "deque": "cpp",
+        "list": "cpp",
+        "unordered_map": "cpp",
+        "vector": "cpp",
+        "exception": "cpp",
+        "algorithm": "cpp",
+        "functional": "cpp",
+        "ratio": "cpp",
+        "system_error": "cpp",
+        "tuple": "cpp",
+        "type_traits": "cpp",
+        "fstream": "cpp",
+        "initializer_list": "cpp",
+        "iomanip": "cpp",
+        "iosfwd": "cpp",
+        "iostream": "cpp",
+        "istream": "cpp",
+        "limits": "cpp",
+        "memory": "cpp",
+        "new": "cpp",
+        "numeric": "cpp",
+        "sstream": "cpp",
+        "stdexcept": "cpp",
+        "streambuf": "cpp",
+        "thread": "cpp",
+        "cfenv": "cpp",
+        "cinttypes": "cpp",
+        "utility": "cpp",
+        "typeindex": "cpp",
+        "typeinfo": "cpp",
+        "string": "cpp",
+        "codecvt": "cpp",
+        "condition_variable": "cpp",
+        "unordered_set": "cpp",
+        "hash_map": "cpp",
+        "hash_set": "cpp",
+        "future": "cpp",
+        "mutex": "cpp",
+        "valarray": "cpp",
+        "*.ipp": "cpp"
+    }
+}
diff --git a/isee-preprocessing/CMakeLists.txt b/isee-preprocessing/CMakeLists.txt
@@ -0,0 +1,73 @@
+cmake_minimum_required(VERSION 2.8.3)
+project(isee_preprocessing)
+set(CMAKE_BUILD_TYPE "Release")
+set(CMAKE_CXX_FLAGS "-std=c++11 -fext-numeric-literals")
+set(CMAKE_CXX_FLAGS_RELEASE "-O2 -Wall -g")
+find_package(catkin REQUIRED COMPONENTS
+  message_generation
+  roscpp
+  rospy
+  std_msgs
+  pcl_conversions
+  pcl_ros
+  cv_bridge
+  image_transport
+  sensor_msgs 
+  geometry_msgs
+  camera_info_manager
+)
+
+find_package( OpenCV 3.2.0 REQUIRED )
+
+## Generate messages in the 'msg' folder
+add_message_files(
+	FILES
+    CustomPoint.msg
+    CustomMsg.msg
+     GPS.msg
+)
+add_service_files(
+  FILES
+  FirmwareUpdate.srv
+  refLLAUpdate.srv
+  )
+
+## Generate added messages and services with any dependencies listed here
+generate_messages(
+    DEPENDENCIES
+    std_msgs  # Or other packages containing msgs
+    geometry_msgs
+)
+
+
+
+include_directories(include ${catkin_INCLUDE_DIRS} ${OpenCV_INCLUDE_DIRS} include /opt/MVS/include/)
+LINK_DIRECTORIES( /opt/MVS/lib/64/ )
+
+add_executable(isee_livox_repub src/livox_repub.cpp)
+target_link_libraries(isee_livox_repub ${catkin_LIBRARIES} ${PCL_LIBRARIES} ${OpenCV_LIBS})
+
+add_executable(imu_processing src/data_process_node.cpp src/data_process.cpp src/gyr_int.cpp )
+target_link_libraries(imu_processing ${catkin_LIBRARIES} ${PCL_LIBRARIES} ${OpenCV_LIBS})
+
+add_executable(isee_synchronize src/isee_synchronize.cpp src/camera.cc)
+add_dependencies(isee_synchronize ${PROJECT_NAME}_generate_messages_cpp)
+target_link_libraries(isee_synchronize ${catkin_LIBRARIES} ${OpenCV_LIBRARIES} ${PCL_LIBRARIES}
+                      GCBase_gcc421_v3_0
+                      MvCameraControl
+                      GenApi_gcc421_v3_0
+                      MVGigEVisionSDK
+                      log4cpp_gcc421_v3_0
+                      MVRender
+                      Log_gcc421_v3_0
+                      MvUsb3vTL
+                      MathParser_gcc421_v3_0
+                      NodeMapData_gcc421_v3_0
+                      MediaProcess
+                      XmlParser_gcc421_v3_0
+                      X11
+                     )
+
+
+
+
diff --git a/isee-preprocessing/build/CATKIN_IGNORE b/isee-preprocessing/build/CATKIN_IGNORE