Skip to content

Multibeam Forward Looking Sonar

Jump to bottom Edit New page
Woensug Choi edited this page Jul 28, 2021 · 79 revisions

Introduction

Previous sonar sensor plugins were based on image processing realms by translating each subpixel (point cloud) of the perceived image to resemble sonar sensors with or without sonar equations (Detailed Review for the previous image-based methods). Here, we have developed a ray-based multibeam sonar plugin to consider the phase and reberveration physics of the acoustic signals providing raw sonar intensity-range data (the A-plot) using the point scattering model. Physical characteristics including time and angle ambiguities and speckle noise are considered. The time and angle ambiguity is a function of the point spread function of the coherent imaging system (i.e., side lobes due to matched filtering and beamforming). Speckle is the granular appearance of an image that is due to many interfering scatterers that are smaller than the resolution limit of the imaging system.

  • Features
    • Physical sonar beam/ray calculation with the point scattering model
      • Generating intensity-range (A-plot) raw sonar data
      • Publishes the data with UW APL's sonar image msg format
    • NVIDIA CUDA core GPU parallelization
      • 10Hz refresh rate with 10m range

ezgif com-video-to-gif(3)

Contents

Background

Characteristics/fidelity/Features

Acoustics characteristics based on sonar equation (SNR = SL-2TL-(NL-DI)+TS)

Sonar equations do not consider phase, reverberations between rays

  • Source level (SL) : predefined by sonar specification
  • Transmission Loss (TL) : path, absorption coefficient with ocean (temp, depth) Currently, typical 0.0354 dB/m constant used, with straight-line path (=distance)
  • Noise Level (NL) : Ocean dependent, (maybe some statistics models exists). Currently, Gaussian noise used.
  • Directivity index (DI) : Sonar transducer specification For our purpose, having sonar resolution from the catalogue provides enough information considering typical characteristics of far-field directivity. Currently, half power width of sinc function used..
  • Target strength (TS) : Object material, incident angle dependent. Currently, 1E-4 reflectivity constant used.

Higher fidelity acoustics characteristics

  • Phase, reverberations (Implemented) : For high frequency, phase overlaps and reverberations of reflected signals would be important which is the case for the sonars (currently, operation frequency 900 kHz).
  • Transmission paths : Transmission paths are not straight-line (=distance). The bellhop model can be discussed. This can matter in far-field searching/mission planning.
  • Target strength, Surface/Bottom loss : Material, incident angle dependent target strength can be used for different objects. Obtainig correct coefficients is difficult requring calculation of reflection/transmission between layers.

Plugin diagram

image

Related works

Work has been published describing the development of a sonar simulation model. Two of these have developed their models into supporting robotics framework.

In Demarco, 2015 a Gazebo sonar sensor model is developed using ray tracing. The Gazebo ray-tracing functionality generates a 3D point cloud which is transformed into a sonar image. On inspection, the acoustic properties were either hard-coded or commented out and did not include speckle noise simulation.

In Cirqueira, 2017, a GPU-based sonar simulator is developed using rasterization. They model two types of sonar: mechanically scanned imaging sonar (MSIS) and FLS. The acoustics features provided in their model are accurate and representative of sound propagation. The sonar sensor model is modeled using a virtual camera and utilizes the following three parameters to render the camera as sonar: pulse distance, echo intensity, and field-of-view. Physical tests were also conducted to compare the simulation results with the physical imaging sonar.

image

Approach

The model is based on a ray-based spatial discretization of the model facets, beampattern appropriate for a line array, and a point scattering model of the echo level. Referenced from Rascon, A. (2020). FORWARD-LOOKING SONAR SIMULATION MODEL FOR ROBOTIC APPLICATIONS (Doctoral dissertation, Monterey, CA; Naval Postgraduate School).

Active sonar equation

image image

Ray based beam sonar model

image image image image image image image

Beam pattern

image image image

References

image

Installation

Clone Repositories

Multibeam sonar plugin repository

The multibeam sonar sensor plugin is at nps_uw_multibeam_sonar repository Make sure to include the repository on the workspace before compiling.

git clone https://github.com/Field-Robotics-Lab/nps_uw_multibeam_sonar.git

Acoustic message repository

Final results are exported as a SonarImage.msg of UW APL's sonar image msg format. Make sure to include the repository on the workspace before compiling.

git clone https://github.com/apl-ocean-engineering/hydrographic_msgs.git

CUDA Library Installation

This plugin demands high computation costs. GPU parallelization is used with Nvidia CUDA Library. A discrete NVIDIA Graphics card is required.

Install Cuda. Install CUDA 11.1 on the host machine (Recommended installation method is to use local run file download link If you find conflicting nvidia driver, remove previous driver and reinstall using downloaded run file) This installation file will install both CUDA 11.1 and the NVIDIA graphics driver 455.32 which is best compatible with CUDA 11.1

If you have already dealt with NVIDIA graphics driver at Ubuntu Software&Updates/Additional drivers to use prepriotary drivers, revert it back to use 'Using X.Org X server' to avoid 'The driver already installed, remove beforehand' kind of msg when you run the installation file.

# Remove nvidia drivers
sudo apt remove nvidia-*
sudo apt autoremove
# Disable nouveau driver
sudo bash -c "echo blacklist nouveau > /etc/modprobe.d/blacklist-nvidia-nouveau.conf"
sudo bash -c "echo options nouveau modeset=0 >> /etc/modprobe.d/blacklist-nvidia-nouveau.conf"
sudo update-initramfs -u
# Reboot
sudo reboot
# Download the run file and run
# wget https://developer.download.nvidia.com/compute/cuda/11.1.1/local_installers/cuda_11.1.1_455.32.00_linux.run
sudo sh cuda_11.1.1_455.32.00_linux.run

Once you are done, you will see something like following msg with nvidia-smi command

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.32.00    Driver Version: 455.32.00    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 105...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   42C    P8    N/A /  N/A |      7MiB /  4040MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Final step is to add paths to the cuda excutables and libraries, add these lines to .bashrc to include them. You may resource it by source ~/.bshrc

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}$ 
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Docker. Use docker file located at Dave repository. Please follow instructions

Quickstart

Launch commands

The repository includes three sonar models (Blueview P900, Blueview M450, Seabat F50)

roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch
roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_m450_nps_multibeam.launch
roslaunch nps_uw_multibeam_sonar sonar_tank_seabat_f50_nps_multibeam.launch

if you see some errors, after CUDA and NVIDIA driver installation, try running catkin_make several times.

When closing the simulation with ctrl+c command, it often hangs waiting for the Gazebo window to terminate. You can force close it using pkill gzclient && pkill gzserver command at the another terminal window.

Video clip

ezgif com-video-to-gif(2)

Sonar image

Live view capture plotted using rqt_image_view defined at Launch file

image

Exported data visualized with MATLAB script

image

Configuration

Parameters for the sonar is configured at each model.sdf file.

Viewport properties

Viewport of the sonar is defined using depth camera parameters configuration including FOV, Clip(Range), nBeams, nRays

  • Field of view (FOV)
<horizontal_fov>1.57079632679</horizontal_fov>
  • Range (clip)
<clip>
  <near>02</near> <!-- optimal 2m-60m -->
  <far>60</far>
</clip>
  • nBeams (width), nRays (height, also act as vertical field of view)
<image>
  <width>512</width>
  <height>114</height>
</image>

Sonar properties

Parameters for sonar calculation are defined also at the model.sdf file including Sonar Freq, Bandwidth, Soundspeed, Source Level.

<sonarFreq>900e3</sonarFreq>
<bandwidth>29.9e3</bandwidth>
<soundSpeed>1500</soundSpeed>
<sourceLevel>220</sourceLevel>
<maxDistance>10</maxDistance>

Plugin properties

Calculation settings including Ray skips, Max distance, writeLog/interval, DebugFlag, Publishing topic names can be changed.

  • maxDistance: Defines max distance of the targets which also defines the length of the signal for each beam. Ideally, it should match with the clip parameter of the depth camera properties.
  • ray skips: Used to reduce calculation time to jump rays to calculate. A total number of rays for each beam is defined at the height parameter of the depth camera properties.
  • sensor gain: Used to specify sensor gain for better visualization.
  • plot scaler: Used to scale the value on the bundled viewer window which is plotted with rqt_image_view package of ROS.
  • writeLog Flag : if on, raw data is saved as csv file at /tmp/ with the rate of writeFrameInterval as SonarRawData_000001.csv. You may use the scripts/plotRawData.m to plot the figures at MATLAB.
  • debugFlag Flag : if on, the calculation time for each frame is printed at the console.
<maxDistance>10</maxDistance>
<raySkips>10</raySkips>
<sensorGain>0.02</sensorGain>
<plotScaler>1</plotScaler>
<writeLog>false</writeLog>
<debugFlag>false</debugFlag>
<writeFrameInterval>5</writeFrameInterval>
<!-- This name is prepended to ROS topics -->
<cameraName>blueview_p900</cameraName>
<!-- ROS publication topics -->
<imageTopicName>image_raw</imageTopicName>
<cameraInfoTopicName>image_raw/camera_info</cameraInfoTopicName>
<pointCloudTopicName>point_cloud</pointCloudTopicName>
<depthImageTopicName>image_depth</depthImageTopicName>
<depthImageCameraInfoTopicName>image_depth/camera_info</depthImageCameraInfoTopicName>
<sonarImageRawTopicName>sonar_image_raw</sonarImageRawTopicName>
<sonarImageTopicName>sonar_image</sonarImageTopicName>
<frameName>forward_sonar_optical_link</frameName>

Variational Reflectivity

Although high fidelity target strength is beyond reach for simple implementation, a user can give different surface reflectivity on the scene's objects.

How it works

  • The parameter to switch the feature is defined at model.sdf of the sonar model (in this case, nps_uw_multibeam_sonar/models/blueview_p900_nps_multibeam/model.sdf 
    • <constantReflectivity>true</constantReflectivity>
<!-- The CSV databsefile is located at the worlds folder -->
<reflectivityDatabaseFile>variationalReflectivityDatabase.csv</reflectivityDatabaseFile>
  • Values are defined for each model object in the world. The values are defined using the CSV file in the worlds folder. The default value is 0.001 
    • basement_tank_model,0.0001
cylinder_target1,0.005
cylinder_target2,0.001
  • cv::Mat this->reflectivityImage is calculated if the scene is changed (it detects the change by comparing the maximum depth value of the depth camera with the previous scene. If the value is stabled (if equal for 3 consecutive frames, the calculation is performed once)
  • The reflectivityImage OpenCV matrix is then transferred to GPU memory and used for sonar calculation

Results

image

Output ROS msg

The final output of the sonar image is sent in two types.

  • Topic name sonar_image
    • This is a msg used internally to plot using with rqt_image_view package of ROS.
    • The data is generated using OpenCV's CV_16UC1 format and changed into msg format using MONO16 format
  • Topic name sonar_image_raw

Benchmarks

Computation time benchmark

  • at workstation (i9-9900K 3.6GHz, Nvidia GeForce RTX 2080Ti)
  • 10 Hz achievable with Ray and Range reduced
512 Beams Refresh
Rate [Hz]		 Total
Time [s]		 Core Sum Corr FFT
Full Calculation
60 m Range
114 Rays		 0.5 Hz 1.7 s 0.3 1.26 0.05 0.03
Ray Reduced
60 m Range
11 Rays		 3.0 Hz 0.27 s 0.02 0.16 0.05 0.03
Ray/Range Reduced
10 m Range
11 Rays		 10 Hz 0.06 s 0.00 0.04 0.01 0.00

Examples

Circular sonar tank, cylinder rod, plate, oculus sonar

  • Launch/world files
    • sonar_tank_oculus_m1200d_nps_multibeam.launch
    • sonar_tank_oculus_m1200d_nps_multibeam.world

image

Acknowledge

  • Woensug Choi
  • Brian Bingham
  • Duane Davis
  • Derek Olsen
  • Bruce Allen
  • Carson Vogt
  • Laura Lindzey
Add a custom footer

Home

Clone this wiki locally