Skip to content

Multibeam Forward Looking Sonar

Jump to bottom Edit New page
Woensug Choi edited this page Dec 13, 2021 · 79 revisions

Introduction

Previous sonar sensor plugins were based on image processing realms by translating each subpixel (point cloud) of the perceived image to resemble sonar sensors with or without sonar equations (Detailed Review for the previous image-based methods). Here, we have developed a ray-based multibeam sonar plugin to consider the phase and reberveration physics of the acoustic signals providing raw sonar intensity-range data (the A-plot) using the point scattering model. Physical characteristics including time and angle ambiguities and speckle noise are considered. The time and angle ambiguity is a function of the point spread function of the coherent imaging system (i.e., side lobes due to matched filtering and beamforming). Speckle is the granular appearance of an image that is due to many interfering scatterers that are smaller than the resolution limit of the imaging system.

summary

  • Note: you may need artificialVehicleVibration tag on to obtain sparkling sonar image

Contents

Background

Characteristics/fidelity/Features

Acoustics characteristics based on sonar equation (SNR = SL-2TL-(NL-DI)+TS)

Sonar equations do not consider phase, reverberations between rays

  • Source level (SL) : predefined by sonar specification
  • Transmission Loss (TL) : path, absorption coefficient with ocean (temp, depth) Currently, typical 0.0354 dB/m constant used, with straight-line path (=distance)
  • Noise Level (NL) : Ocean dependent, (maybe some statistics models exists). Currently, Gaussian noise used.
  • Directivity index (DI) : Sonar transducer specification For our purpose, having sonar resolution from the catalogue provides enough information considering typical characteristics of far-field directivity. Currently, half power width of sinc function used..
  • Target strength (TS) : Object material, incident angle dependent. Currently, 1E-4 reflectivity constant used.

Higher fidelity acoustics characteristics

  • Phase, reverberations (Implemented) : For high frequency, phase overlaps and reverberations of reflected signals would be important which is the case for the sonars (currently, operation frequency 900 kHz).
  • Transmission paths : Transmission paths are not straight-line (=distance). The bellhop model can be discussed. This can matter in far-field searching/mission planning.
  • Target strength, Surface/Bottom loss : Material, incident angle dependent target strength can be used for different objects. Obtainig correct coefficients is difficult requring calculation of reflection/transmission between layers.

Plugin diagram

diagram

Related works

Work has been published describing the development of a sonar simulation model. Two of these have developed their models into supporting robotics framework.

In Demarco, 2015 a Gazebo sonar sensor model is developed using ray tracing. The Gazebo ray-tracing functionality generates a 3D point cloud which is transformed into a sonar image. On inspection, the acoustic properties were either hard-coded or commented out and did not include speckle noise simulation.

In Cirqueira, 2017, a GPU-based sonar simulator is developed using rasterization. They model two types of sonar: mechanically scanned imaging sonar (MSIS) and FLS. The acoustics features provided in their model are accurate and representative of sound propagation. The sonar sensor model is modeled using a virtual camera and utilizes the following three parameters to render the camera as sonar: pulse distance, echo intensity, and field-of-view. Physical tests were also conducted to compare the simulation results with the physical imaging sonar.

comparisons

Approach

The model is based on a ray-based spatial discretization of the model facets, beampattern appropriate for a line array, and a point scattering model of the echo level.

Single beam sonar model

beammodel1 beammodel2

Ray based beam sonar model

raymodel1 raymodel2 raymodel3

Beam pattern

directivity1 directivity2

Overall procedures

algorithm procedures

References

reference

Installation

Clone Repositories

Option A. Using vcs tool

cd ~/uuv_ws/src
vcs import --skip-existing --input dave/extras/repos/multibeam_sim.repos .

vcs will use multibeam_sim.repos file inthe dave repository that includes both nps_uw_multibeam_sonar and hydrographic_msgs.

Option B. Git clone manually

Multibeam sonar plugin repository

The multibeam sonar sensor plugin is at nps_uw_multibeam_sonar repository Make sure to include the repository on the workspace before compiling.

git clone https://github.com/Field-Robotics-Lab/nps_uw_multibeam_sonar.git

Acoustic message repository

Final results are exported as a SonarImage.msg of UW APL's sonar image msg format. Make sure to include the repository on the workspace before compiling.

git clone https://github.com/apl-ocean-engineering/hydrographic_msgs.git

CUDA Library Installation

This plugin demands high computation costs. GPU parallelization is used with Nvidia CUDA Library. A discrete NVIDIA Graphics card is required.

Option A. Use Docker

The simplest way to pepare your machine with CUDA library would be to use Docker environment. Following commends include -b cuda which includes cuda library.

git clone -b cuda https://github.com/Field-Robotics-Lab/dockwater.git
cd dockwater
./build.bash noetic
./run.bash noetic:latest

When docker environment is ready, follow this link to compile : Build Dave Environment

Option B. Install on the Host

Install Cuda. Install CUDA 11.1 on the host machine (Recommended installation method is to use local run file download link If you find conflicting nvidia driver, remove previous driver and reinstall using downloaded run file) This installation file will install both CUDA 11.1 and the NVIDIA graphics driver 455.32 which is best compatible with CUDA 11.1

If you have already dealt with NVIDIA graphics driver at Ubuntu Software&Updates/Additional drivers to use prepriotary drivers, revert it back to use 'Using X.Org X server' to avoid 'The driver already installed, remove beforehand' kind of msg when you run the installation file.

# Remove nvidia drivers
sudo apt remove nvidia-*
sudo apt autoremove
# Disable nouveau driver
sudo bash -c "echo blacklist nouveau > /etc/modprobe.d/blacklist-nvidia-nouveau.conf"
sudo bash -c "echo options nouveau modeset=0 >> /etc/modprobe.d/blacklist-nvidia-nouveau.conf"
sudo update-initramfs -u
# Reboot
sudo reboot
# Download the run file and run
# wget https://developer.download.nvidia.com/compute/cuda/11.1.1/local_installers/cuda_11.1.1_455.32.00_linux.run
sudo sh cuda_11.1.1_455.32.00_linux.run

Once you are done, you will see something like following msg with nvidia-smi command

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.32.00    Driver Version: 455.32.00    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 105...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   42C    P8    N/A /  N/A |      7MiB /  4040MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Final step is to add paths to the cuda excutables and libraries, add these lines to .bashrc to include them. You may resource it by source ~/.bshrc

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}$
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Quickstart

Launch commands

The repository includes three sonar models (Blueview P900, Blueview M450, Seabat F50)

roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch
roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_m450_nps_multibeam.launch
roslaunch nps_uw_multibeam_sonar sonar_tank_seabat_f50_nps_multibeam.launch

if you see some errors, after CUDA and NVIDIA driver installation, try running catkin_make several times.

When closing the simulation with ctrl+c command, it often hangs waiting for the Gazebo window to terminate. You can force close it using pkill gzclient && pkill gzserver command at the another terminal window.

Video clip

ezgif com-video-to-gif(2)

  • Note: you may need artificialVehicleVibration tag on to obtain sparkling sonar image

Sonar image

Live view capture plotted using rqt_image_view defined at Launch file

image

Exported data visualized with MATLAB script

image

Configuration

Parameters for the sonar is configured at each model.sdf file.

Viewport properties

Viewport of the sonar is defined using depth camera parameters configuration including FOV, Clip(Range), nBeams, nRays

  • Field of view (FOV)
<!-- 90 degrees for the M900-90 -->
<horizontal_fov>1.57079632679</horizontal_fov>
  • Range (clip)
<clip>
  <near>02</near> <!-- optimal 2m-60m -->
  <far>60</far>
</clip>
  • nBeams (width), nRays (height, also act as vertical field of view)
<image>
  <width>512</width>
  <!-- Set vertical FOV by setting image height -->
  <!-- Approx 2 times the spec sheet (sepc: 20 deg.) -->
  <height>224</height>
</image>
  • Here, the <height> is set to define as if 40 deg which is approximately two times that of the spec sheet. The correct vertical FOV is defined at <verticalFOV> below.

Sonar properties

Parameters for sonar calculation are defined also at the model.sdf file including Sonar Freq, Bandwidth, Soundspeed, Source Level.

<verticalFOV>20</verticalFOV>
<sonarFreq>900e3</sonarFreq>
<bandwidth>29.9e3</bandwidth>
<soundSpeed>1500</soundSpeed>
<sourceLevel>220</sourceLevel>
<maxDistance>10</maxDistance>

Plugin properties

Calculation settings including Ray skips, Max distance, writeLog/interval, DebugFlag, Publishing topic names can be changed.

  • maxDistance: Defines max distance of the targets which also defines the length of the signal for each beam. Ideally, it should match with the clip parameter of the depth camera properties.
  • ray skips: Used to reduce calculation time to jump rays to calculate. A total number of rays for each beam is defined at the height parameter of the depth camera properties.
  • sensor gain: Used to specify sensor gain for better visualization.
  • plot scaler: Used to scale the value on the bundled viewer window which is plotted with rqt_image_view package of ROS.
  • writeLog Flag : if on, raw data is saved as csv file at /tmp/ with the rate of writeFrameInterval as SonarRawData_000001.csv. You may use the scripts/plotRawData.m to plot the figures at MATLAB.
  • debugFlag Flag : if on, the calculation time for each frame is printed at the console.
  • artificialVehicleVibration Flag : if on, the gaussian noise values will change constantly as sparkling sonar image in the example gif on this wiki. The sparkling noise (continuous change of random noise values even when the sonar scene is static) does not occur in the real world unless the vehicle the sonar is attached is moved or the object in the scene is changed. These physical characteristics are exploited in some scenarios to recognize the changes in the scene by detecting those noise changes. In the plugin, the Gaussian noise value (the random noise value) is changed whenever the maximum distance of the objects in the scene is changed. This flag parameter changes the random values on every frame to imitate as if the vehicle is vibrating in position.
<maxDistance>10</maxDistance>
<raySkips>10</raySkips>
<sensorGain>0.02</sensorGain>
<plotScaler>1</plotScaler>
<writeLog>false</writeLog>
<debugFlag>false</debugFlag>
<writeFrameInterval>5</writeFrameInterval>
<artificialVehicleVibration>false</artificialVehicleVibration>
<!-- This name is prepended to ROS topics -->
<cameraName>blueview_p900</cameraName>
<!-- ROS publication topics -->
<imageTopicName>image_raw</imageTopicName>
<cameraInfoTopicName>image_raw/camera_info</cameraInfoTopicName>
<pointCloudTopicName>point_cloud</pointCloudTopicName>
<depthImageTopicName>image_depth</depthImageTopicName>
<depthImageCameraInfoTopicName>image_depth/camera_info</depthImageCameraInfoTopicName>
<sonarImageRawTopicName>sonar_image_raw</sonarImageRawTopicName>
<sonarImageTopicName>sonar_image</sonarImageTopicName>
<frameName>forward_sonar_optical_link</frameName>

Variational Reflectivity

Although high fidelity target strength is beyond reach for simple implementation, a user can give different surface reflectivity on the scene's objects.

  • Note
    • Variational reflectivity can slow down the refresh rate significantly.
    • The plugin will initiate with constant reflectivity and may take some seconds to refresh the image with variational reflectivities.

Reflectivity by model names

How it works

  • The parameter to switch the feature is defined at model.sdf of the sonar model (in this case, nps_uw_multibeam_sonar/models/blueview_p900_nps_multibeam/model.sdf. Assining the <constantReflectivity> and <customSDFTagReflectivity> as false will turn the variational reflectivity by the model name feature on. 
    • <constantReflectivity>false</constantReflectivity>
<customSDFTagReflectivity>false</customSDFTagReflectivity>
<!-- The CSV databsefile is located at the worlds folder -->
<reflectivityDatabaseFile>variationalReflectivityDatabase.csv</reflectivityDatabaseFile>
  • Values are defined for each model object in the world. The values are defined using the CSV file in the worlds folder. The default value is 0.001. The example CSV file is at /worlds/variationalReflectivityDatabase.csv 
    • basement_tank_model,0.0001
cylinder_target1,0.005
cylinder_target2,0.001
  • cv::Mat this->reflectivityImage is calculated if the scene is changed (it detects the change by comparing the maximum depth value of the depth camera with the previous scene. If the value is stabled (if equal for 3 consecutive frames, the calculation is performed once)
  • The reflectivityImage OpenCV matrix is then transferred to GPU memory and used for sonar calculation

Results

image

Reflectivity by custom SDF tags

Quickstart

roslaunch nps_uw_multibeam_sonar sonar_tank_oculus_m1200d_nps_multibeam_customSDFTag.launch

How it works

  • The parameter to switch the feature is defined at model.sdf of the sonar model (in this case, nps_uw_multibeam_sonar/models/blueview_p900_nps_multibeam/model.sdf. Assining the <constantReflectivity> as false and <customSDFTagReflectivity> as true will turn the variational reflectivity by the custom Tag feature on.
  • Example tag names and how they are defined 
    • <!-- Custom SDF elements for surface properties -->
<surface_props:material>metal</surface_props:material>
<surface_props:biofouling_rating>10</surface_props:biofouling_rating>
<surface_props:roughness>0.1</surface_props:roughness>
  • You can add as many tags as you want. How they contribute to the final reflectivity is defined at the CSV file (/worlds/customSDFTagDatabase.csv) 
    • biofouling_rating,20.0
roughness,5.0
default,0.001
metal,0.005
rubber,0.0001
    • the material name (default, metal, and rubber) defines the base reflectivity values. The biofouling_rating, roughness, and any other tags you want to add will affect the reflectivity value as coefficients to change the base value.

Results

image

Output ROS msg

The final output of the sonar image is sent in two types.

  • Topic name sonar_image
    • This is a msg used internally to plot using with rqt_image_view package of ROS.
    • The data is generated using OpenCV's CV_16UC1 format and changed into msg format using MONO16 format
  • Topic name sonar_image_raw

Benchmarks

Computation time benchmark

  • at workstation (i9-9900K 3.6GHz, Nvidia GeForce RTX 2080Ti)
  • 10 Hz achievable with Ray and Range reduced
512 Beams Refresh
Rate [Hz]		 Total
Time [s]		 Core Sum Corr FFT
Full Calculation
60 m Range
114 Rays		 0.5 Hz 1.7 s 0.3 1.26 0.05 0.03
Ray Reduced
60 m Range
11 Rays		 3.0 Hz 0.27 s 0.02 0.16 0.05 0.03
Ray/Range Reduced
10 m Range
11 Rays		 10 Hz 0.06 s 0.00 0.04 0.01 0.00

Acknowledge

  • Woensug Choi
  • Brian Bingham
  • Duane Davis
  • Derek Olsen
  • Bruce Allen
  • Carson Vogt
  • Laura Lindzey
Add a custom footer

Home

Clone this wiki locally