[Config Support]: No audio when using rtsp restream in HAOS

[hapklaar]

Describe the problem you are having

I run Frigate in a separate LXC container, connect to the camera using the go2rtc method and use the restream url in Home Assistant. This works fine, except that I don't have audio when using the WebRTC card in Home Assistant is in webrtc mode. When I set it to mse mode, audio works, but the stream will freeze on iOS (known issue). So I'm stuck with webrtc for now.

When I connect to the camera from HASS, I do have audio in the RTSP stream in webrtc mode.

The issue seems similar to #6214, but in my case Frigate doesn't run as a HAOS addon.

Version

0.13.2-6476F8A

Frigate config file

mqtt:
  enabled: true
  host: 192.168.111.201
  user: user
  password: pass

ffmpeg:
  hwaccel_args: preset-vaapi

detectors:
  ov:
    type: openvino
    device: GPU
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

snapshots:
  enabled: True
  bounding_box: True
  timestamp: False

birdseye:
  enabled: false
  mode: continuous

go2rtc:
  streams:
    Voor:
    - rtsp://admin:[email protected]:554/h264Preview_01_main
    - ffmpeg:Voor#audio=opus

cameras:
  Voor:
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
      - path: rtsp://127.0.0.1:8554/Voor
        input_args: preset-rtsp-restream
        roles:
        - record
        - detect
    detect:
      enabled: true
      width: 1152
      height: 864
      fps: 20
    objects:
      track:
        - person
        - cat
        - dog
    record:
      enabled: true
      retain:
        days: 2
        mode: all
      events:
        retain:
          default: 3
          mode: motion
    zones:
      zone_0:
        coordinates: 0,1920,2560,1920,2560,905,0,961
    mqtt:
      timestamp: false
      bounding_box: false
      crop: true
      quality: 100
      height: 500

Relevant log output

nvt

Frigate stats

No response

Operating system

Debian

Install method

Docker Compose

Coral version

CPU (no coral)

Any other information that may be helpful

Pasting working and non-working WebRTC card config here:

No audio:

type: custom:webrtc-camera
url: rtsp://192.168.111.102:8554/Voor
mode: webrtc

With audio:

type: custom:webrtc-camera
url: rtsp://192.168.111.102:8554/Voor
mode: mse

[NickM-27]

I would suggest using the recommended reolink config https://docs.frigate.video/configuration/camera_specific#reolink-cameras

also, the detect fps does not need to match the camera fps and generally 5-10 is the recommended range

[hapklaar]

currently this:

mqtt:
  enabled: true
  host: 192.168.111.201
  user: user
  password: pass

ffmpeg:
  hwaccel_args: preset-vaapi

detectors:
  ov:
    type: openvino
    device: GPU
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

snapshots:
  enabled: True
  bounding_box: True
  timestamp: False

birdseye:
  enabled: false
  mode: continuous

go2rtc:
  streams:
    Voor:
    - ffmpeg:https://192.168.111.70/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=password#video=copy#audio=copy#audio=opus

cameras:
  Voor:
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
      - path: rtsp://127.0.0.1:8554/Voor
        input_args: preset-rtsp-restream
        roles:
        - record
        - detect
    detect:
      enabled: true
      width: 1152
      height: 864
      fps: 10
    objects:
      track:
        - person
        - cat
        - dog
    record:
      enabled: true
      retain:
        days: 2
        mode: all
      events:
        retain:
          default: 3
          mode: motion
    zones:
      zone_0:
        coordinates: 0,1920,2560,1920,2560,905,0,961
    mqtt:
      timestamp: false
      bounding_box: false
      crop: true
      quality: 100
      height: 500
      

[NickM-27]

you need to setup webrtc https://docs.frigate.video/configuration/live#webrtc-extra-configuration

[hapklaar]

Added config as such:

go2rtc:
  streams:
    Voor:
    - ffmpeg:https://192.168.111.70/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=password#video=copy#audio=copy#audio=opus
    webrtc:
    candidates:
      - 192.168.111.102:8555
      - stun:8555

also opened the port in the docker compose file, can see it initialising in go2rtc logs:
17:42:58.059 INF [webrtc] listen addr=:8555
am able to telnet to the port

Tried both rtsp://192.168.111.102:8554/Voor and rtsp://192.168.111.102:8555/Voor in the WebRTC card. Could only connect using port 8554 but then issue remains. Getting a timeout on port 8555.

Should I use port 8555 now for mode webrtc in Home Assistant?

[NickM-27]

candidates is not indented correctly

[NickM-27]

the webrtc card needs to be setup to connect to frigate:1984 for it to be able to access this directly