Skip to content

Latest commit

 

History

History
117 lines (67 loc) · 4.47 KB

README.rst

File metadata and controls

117 lines (67 loc) · 4.47 KB
Raw

Multi-Stream object detection Pipeline

Overview

This GStreamer pipeline demonstrates object detection on multiple camera streams over RTSP protocol / Files.

All the streams are processed in parallel through the decode and scale phases, and enter the Hailo device frame by frame.

Afterwards postprocess and drawing phases add the classified object and bounding boxes to each frame. The last step is to match each frame back to its respective stream and output all of them to the display.

Read more about RTSP: RTSP

Prerequisites

Preparations

In case of using RTSP cameras, configuration of the RTSP camera sources is required before running. open the multi_stream_detection_rtsp.sh in edit mode with your preferred editor. Configure the eight sources to match your own cameras.

readonly SRC_0="rtsp://<ip address>/?h264x=4 user-id=<username> user-pw=<password>"
readonly SRC_1="rtsp://<ip address>/?h264x=4 user-id=<username> user-pw=<password>"
etc..

Run the pipeline

./multi_stream_detection.sh

OR

./multi_stream_detection_rtsp.sh

  1. --show-fps Prints the fps to the output.

  2. --num-of-sources Sets the number of sources to use by given input. The default and recommended value in this pipeline is 12 (for files) or 8 (for camera streams over RTSP protocol)"

  3. --debug Uses gst-top to print time and memory consuming elements, saves the results as text and graph.

    NOTE: : When the debug flag is used and the app is running inside of a docker, exit the app by tying Ctrl+C in order to save the results. (Due to docker X11 display communication issues)

The output should look like:

readme_resources/example.jpg

Configuration

The app post process parameters can be configured by a json file located in $TAPPAS_WORKSPACE/apps/gstreamer/general/multistream_detection/resources/configs/yolov5.json

Supported Networks

Overview of the pipeline

These apps are based on our multi stream pipeline template

RTSP specific elements used

  • rtspsrc Makes a connection to an rtsp server and read the data. Used as a src to get the video stream from rtsp-cameras.
  • rtph264depay Extracts h264 video from rtp packets.

HailoRT Stream Multiplexer example

  • This app shows the usage of the HailoRT Stream Multiplexer. This feature controls the time shared on the Hailo device between all streams. The Stream Multiplexer is enabled by the Hailonet scheduling-algorithm property when in use in multiple Hailonet elements that run the same HEF file. When the Stream Multiplexer is in use, there is no need to use funnel and streamiddemux like elements because the logic is handeled internally.

How to use Retraining to replace models

Note

It is recommended to first read the Retraining TAPPAS Models page.

You can use Retraining Dockers (available on Hailo Model Zoo), to replace the following models with ones that are trained on your own dataset:

  • yolov5m
    • Retraining docker
      • For best compatibility and performance with TAPPAS, use for compilation the corresponsing YAML file from above.
      • Should use ModelZoo to compile together with centerpose for this pipeline. See detection_pose_estimation.yaml
    • TAPPAS changes to replace model:
      • Update HEF_PATH on the .sh file
      • Update resources/configs/yolov5.json with your new post-processing parameters (NMS)