USB Frigate inference time ... 30-60 ms then drops to approx. 10 ms

[ElectronicBattle]

Hello

Question:
Why does my inference time change when I start using the UI and what is poor about my config which I could alter in order to get the inference time back to its original low value?

Background:
My eight cameras were processed in 0.13 with consistently short (10-20 ms) inference times as per the title. I have recently u/g to 0.14 and then 0.14.1 and in doing so I have adjusted my config and tried a few ideas. Something has changed because

• 0.13 to 0.14: first the inference time rose to be four or five times slower

• then overnight, with (I think) no changes, it settled down and remained at 10 ms or so (IR greyscale images, lower data rates, fewer motion/object activations/detections? Or my imagination and I did change things but forgot ...)

• 0.14 and also 0.14 to 0.14.1: I fiddled and messed up the config such that everything is working but the inference time has again changed to those higher figures

• I now see that when Frigate is running and left alone the inference time is short but when I start using the webUI to look at things, it rises (please see metrics screenshot below).

In amongst these config alterations I have lost track of a couple of changes (I ought to try to work out a decent version control system rather than my memory) and now I can't see how to get back to where I was.

It's all working beautifully so I suppose I oughtn't to worry, but returning to the days of constantly <15 ms inference time is something I would like to do before I start experimenting with a new graphics card.

Here's my "slow" config for 0.14.1:

#BASELINE8_150125_0.14.1 based on but updated for version
#BASELINE8_170824_013 this works OK on Frigate and on frigateviewer: 9ms inference and <50% TN CPU load fastinference8cameras013_config_310724_NO_RESTREAMING_FIXED taken from #config_and_audio_110724# # this works with the #FrigateViewer tablet app - events playback OK
# improved detection (height & width added) # also added ffmpeg out args record audio and logger entries at EOF

objects:
  filters:
    dog:
      min_score: .8
      threshold: .9
    cat:
      min_score: .65
      threshold: .8
    face:
      min_score: .7
    package:
      min_score: .65
      threshold: .9
    license_plate:
      min_score: .6
    amazon:
      min_score: .75
    ups:
      min_score: .75
    fedex:
      min_score: .75
    person:
      min_score: .65
      threshold: .85
    car:
      min_score: .65
      threshold: .85


ui:
  time_format: 24hour

mqtt:
  enabled: false

model:
  path: plus://e570a5fa60ccfce5aca76ad55c4ffddd  #mobiledet edgeTPU 2024.3 7/11/24



go2rtc:
  streams:
    #reference format from NickM-27 @ https://github.com/blakeblackshear/frigate/discussions/12376 
    #              "ffmpeg:rtsp://...                                               #video=h264#audio=copy#hardware" note the quotes and hashes
    greenhouse:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=0&level=0/track0#video=h264#audio=copy
      - ffmpeg:greenhouse#audio=opus
    upper_patio:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=1&level=0/track0#video=h264#audio=copy
      - ffmpeg:upper_patio#audio=opus
    ponds:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=2&level=0/track0#video=h264#audio=copy
      - ffmpeg:ponds#audio=opus
    blue_gate:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=3&level=0/track0#video=h264#audio=copy
      - ffmpeg:blue_gate#audio=opus
    lower_steps:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=4&level=0/track0#video=h264#audio=copy
      - ffmpeg:lower_steps#audio=opus
    lower_patio:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=5&level=0/track0#video=h264#audio=copy
      - ffmpeg:lower_patio#audio=opus
    shed:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=6&level=0/track0#video=h264#audio=copy
      - ffmpeg:shed#audio=opus
    drive:
      - ffmpeg:rtsp://192.168.1.123:554/media=0/channel=7&level=0/track0#video=h264#audio=copy
      - ffmpeg:drive#audio=opus
#    doorbell:
#      - "ffmpeg:rtsp://admin:[email protected]:554/Streaming/Channels/101#raw -vf lenscorrection=k1=-0.450:k2=0.1"
#      - "ffmpeg:doorbell#audio=opus"

####rtsp is 30060 and webRTC is 30061 (TN Frigate app interrnals)
  webrtc:
    candidates:
      - 192.168.1.194:30061
      - stun:30061


ffmpeg:
  #####you can only have one line here#####
  #hwaccel_args: preset-nvidia-h264
  #hwaccel_args: preset-nvidia-hevc #gives black screens on cameras
  #hwaccel_args: preset-nvidia-cuvid #gives black screens on cameras
  output_args:
    record: preset-record-generic-audio-aac

detectors:
  coral:
    type: edgetpu
    device: usb
    detect:
      enabled: true

birdseye:
  enabled: true
  mode: continuous #continuous | objects | motion

# record: # <----- Enable recording # was working
#  enabled: true # was working

# added from https://fixtse.com/blog/coral-ia-frigate start#
record:
  enabled: true
  retain:
    days: 4
    mode: motion
  events:
    retain:
      default: 7
      mode: active_objects
# added from https://fixtse.com/blog/coral-ia-frigate #

snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  enabled: true
  # Optional: save a clean PNG copy of the snapshot image (default: shown below)
  clean_copy: true #false  #true
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: false
  # Optional: draw bounding box on the snapshots (default: shown below)
  bounding_box: true
  # Optional: crop the snapshot (default: shown below)
  crop: false
  # Optional: height to resize the snapshot to (default: original size)
  height: 768
  #height: 800
  # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
  required_zones: []
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 3
    # Optional: Per object retention days
    objects:
      person: 7
      cat: 4
      dog: 4
      bird: 1
      fox: 4
      badger: 4
      car: 7
      license_plate: 7

  # Optional: quality of the encoded jpeg, 0-100 (default: shown below)
  quality: 100

##########################################################
##########################################################
# go2rtc: # this worked from https://docs.frigate.video/configuration/live/#setting-stream-for-live-ui
#  streams:
#    upper_patio:
#    - rtsp://192.168.1.123:554/media=0/channel=1&level=0/track0 # <- stream which supports video & aac audio.
#    - "ffmpeg:upper_patio#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
#    restream: true        
#    upper_patio_sub:
#    - rtsp://192.168.1.123:554/media=0/channel=1&level=1/track0 # <- stream which supports video & aac audio.
#    - "ffmpeg:upper_patio_sub#audio=opus"
##########################################################
##########################################################

motion:
  improve_contrast: true

detect:
  width: 640
  height: 360

#record:
#  width: 1920
#  height: 1080

########################################################################################################################################################################################################################################

cameras: #########GREENHOUSE##################GREENHOUSE##################GREENHOUSE##################GREENHOUSE##################GREENHOUSE#########GREENHOUSE#########
  greenhouse:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=0&level=1/track0 #level1 = SD 640x360 (checked CORRECT); track0 media=0 are the only working options tested in ffplay
          roles:
            - detect


        - path: rtsp://192.168.1.123:554/media=0/channel=0&level=0/track0 #level0 = HD 1920x1080(checked CORRECT);  track0 media=0 are the only working options tested in ffplay
          roles:
            - record

    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placeholder
        - badger #placeholder

    birdseye:
      mode: continuous #continuous | objects | motion
    # masks are black: they block out areas where you aren't interested in detecting things
      order: 1




  upper_patio: #########UPPER_PATIO###############################################UPPER_PATIO###########################UPPER_PATIO###########################UPPER_PATIO###########################
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=1&level=1/track0 #level1 = SD 640x360 (checked CORRECT); track0 media=0 are the only working options tested in ffplay
          roles:
            - detect


        - path: rtsp://192.168.1.123:554/media=0/channel=1&level=0/track0 #level0 = HD 1920x1080(checked CORRECT);  track0 media=0 are the only working options tested in ffplay
          roles:
            - record

    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placeholder
        - badger #placeholder

    birdseye:
      mode: continuous #continuous | objects | motion
    # masks are black: they block out areas where you aren't interested in detecting things
      order: 2

    motion:
      mask: 
        0,219,72,169,140,139,146,79,218,73,291,72,362,133,430,143,450,121,488,120,523,98,640,95,640,0,0,0

    zones:
      upper patio:
        coordinates: 
          640,360,640,337,610,312,603,297,594,279,564,256,524,255,510,242,487,220,451,203,435,201,421,187,409,166,392,152,334,146,274,138,215,142,122,145,78,162,23,202,0,360




  ponds: #################PONDS#######################################PONDS#########################PONDS#########################PONDS#########################PONDS#########################
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=2&level=1/track0 #level1 = SD 640x360 (checked CORRECT); track0 media=0 are the only working options tested in ffplay
          roles:
            - detect


        - path: rtsp://192.168.1.123:554/media=0/channel=2&level=0/track0 #level0 = HD 1920x1080(checked CORRECT);  track0 media=0 are the only working options tested in ffplay
          roles:
            - record
    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placeholder
        - badger #placeholder

    birdseye:
      mode: continuous  #continuous | objects | motion
     # masks are black: they block out areas where you aren't interested in detecting things
      order: 3



  blue_gate: #########BLUE_GATE###################################################BLUE_GATE#############################BLUE_GATE#############################BLUE_GATE#############################BLUE_GATE#############################
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=3&level=1/track0 #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=3&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
      #  input_args: preset-rtsp-restream

          roles:
            - detect
        - path: rtsp://192.168.1.123:554/media=0/channel=3&level=0/track0 #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
          roles:
            - record

    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placehlder
        - badger # placeholder
        - car
        - license_plate

    birdseye:
      mode: continuous   #continuous | objects | motion
      # masks are black: they block out areas where you aren't interested in detecting things
      order: 4

    motion:
      mask:
        - 0,153,85,148,140,139,239,140,300,142,354,145,398,141,429,141,486,145,518,213,450,226,605,360,640,360,640,0,0,0
        - 479,360,423,337,411,310,390,288,369,268,352,248,340,228,297,223,267,247,231,253,212,204,188,200,152,218,107,227,88,222,73,234,63,250,54,266,45,282,27,292,0,287,0,360

    zones:
      gravel:
        coordinates: 
          411,273,471,360,523,360,585,360,474,227,473,151,388,141,283,146,228,142,0,151,0,301,296,233



  lower_steps: #########LOWER_STEPS#########LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=4&level=1/track0 #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=5&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
       #input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://192.168.1.123:554/media=0/channel=4&level=0/track0 #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
          roles:
            - record

    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placeholder
        - badger #placeholder

    birdseye:
      mode: continuous   #continuous | objects | motion
      order: 5


  lower_patio: #########LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=5&level=1/track0 #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=5&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
       #input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://192.168.1.123:554/media=0/channel=5&level=0/track0 #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
          roles:
            - record

    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placeholder
        - badger #placeholder

    birdseye:
      mode: continuous   #continuous | objects | motion
      order: 6

      # masks are black: they block out areas where you aren't interested in detecting things
    motion:
      mask:
        - 0,148,92,104,184,55,253,0,269,154,272,305,318,125,354,107,345,56,406,60,417,131,504,240,570,360,640,360,640,0,0,0
        #- 0,219,72,169,140,139,146,79,218,73,291,72,362,133,430,143,450,121,488,120,523,98,640,95,640,0,0,0

    zones:
      lower patio:
        coordinates: 
          284,360,303,183,333,157,344,139,357,118,352,99,359,74,382,73,384,110,403,128,499,246,580,360




  shed: #########SHED##########################SHED#########################SHED#########################SHED#########################SHED#########################SHED#########################
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=6&level=1/track0 #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=5&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
       #input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://192.168.1.123:554/media=0/channel=6&level=0/track0 #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
          roles:
            - record

    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placeholder
        - badger #placeholder

    birdseye:
      mode: continuous   #continuous | objects | motion
      order: 7



  drive: #########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE
    enabled: true

    objects:
      track:
        - face
        - person
        - cat
        - dog
        - bird
        - deer
        - fox #placeholder
        - badger #placeholder
        - car
        - license_plate

    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.123:554/media=0/channel=7&level=1/track0 #level1 = SD; changing track0 to track1 didn't work)
      #input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://192.168.1.123:554/media=0/channel=7&level=0/track0 #level0 = HD ;(changing track0 to track1 didn't work)
          roles:
            - record

    birdseye:
      mode: continuous #continuous | objects | motion
      #restream: True
      order: 8

    motion:
      # masks are black: they block out areas where you aren't interested in detecting things
      mask:
        - 98,129,170,113,189,0,0,0,0,0,0,142
        - 273,0,170,0,178,67,258,117,256,191,414,181,640,307,640,0

    # zones are red: they select areas where monitoring of activity is wanted
    zones:
      driveway:
        coordinates: 
          78,360,248,360,381,360,640,360,640,276,449,177,303,183,298,129,200,64,144,113,0,127,0,360

#  doorbell: #####DOORBELL##########################DOORBELL##########################DOORBELL##########################DOORBELL#####################DOORBELL#####################DOORBELL#####################
#    enabled: true #remember that EZViz won't stream to more than thee (four?) destinations
#    ffmpeg:
#      inputs:
#        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101 # H264 1536x1536
#          roles:
#            - detect
#      output_args:
#        detect: -vf lenscorrection=k1=-0.450:k2=0.1 -f rawvideo -pix_fmt yuv420p
#        - path: rtsp://127.0.0.1:8554/doorbell
#          roles:
#            - record
#      output_args:
#          detect: -vf lenscorrection=k1=-0.450:k2=0.1
#         
#    objects:
#      track:
#        - person
#        - cat
#        - dog
#        - bird
#        - deer
#        - fox #placeholder
#        - badger #placeholder
#    birdseye:
#      mode: continuous     #continuous | objects | motion
#        # masks are black: they block out areas where you aren't interested in detecting things
#      order: 9

# for the doorbell camera. this is what ffplay said (once I'd set the lens correction):

#$  ffplay -i rtsp://admin:[email protected]:554/Streaming/Channels/101 -vf lenscorrection=k1=-0.450:k2=0.095
#  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1536x1536, 15 fps, 15 tbr, 90k tbn
#  Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp



logger:
  logs:
      #frigate.record.maintainer: debug
    frigate.object_detection: debug
      #watchdog.blue_gate: debug


#version: 0.14
#camera_groups:
#  birdseye:
#    order: 1
#    icon: LuActivity
#    cameras: birdseye
version: 0.14

and here's the metrics page:

which shows about 7 ms until 6:40pm this evening which is when i clicked on the browser tab to look at the day's events in frigate, and the amber bar occurs (60 ms) followed by the greens at 40+ ms.

[NickM-27]

This is likely just a CPU issue, when the server has to spend time doing other operations it is struggling to keep the USB running at full speed.

[ElectronicBattle]

I am not sure that CPU load/USB management is the (sole) cause of what I see. This morning:

Prior to 0830 my server was running with no tasks, having been doing nothing overnight.
At 0815 I used a tablet to interrogate Frigate's UI and saw 30 ms inference time
At 0830 I switched on my main PC and tabbed to the browser page from which the above screenshot is taken and saw a nominal 30 ms throughout.

I didn't see any change in Coral inference time at these junctures. I would have expected low speeds (<10 ms) prior to 0815.

Also, looking at my original post, there was nothing loading Frigate (or its host) prior to that step change in inference time, at least not that I can account for.

I haven't changed my config; I'll check again later and overnight to see if i can see this change again.

[ElectronicBattle]

This evening was more of the same: I came home from work (ergo the server was largely dormant during the day, nobody is using it) and looked at the Frigate metrics page which showed the period leading upto the page load, and then after the page load, to be the same 40-plus milliseconds again. I'm not seeing any change in server load correspond to Frigate use (such as me requesting a Frigate web page) so I don't think that that is responsible for USB speed changes.

[NickM-27]

you may try unplugging and replugging, perhaps it is locked at USB 2 speeds for some reason

[ElectronicBattle]

That was a good idea which I should have thought of myself - thanks.
The inference time is now back to about 8 ms; something happened such that the "container" (in inverted commas because it is a TrueNAS app and it is likely to take after Docker in some specific way) needed to be restarted a couple of times.

Now the CPU load for the Frigate app is 10-11% and the inference time is staying low, so I will see what happens.

(This is merely an interest for me: Frigate 0.14.1 continues to work on TN even though the reported Coral inference times seem quite long at times).

[ElectronicBattle]

but a few hours of ordinary uniterrupted running later, and the inference time has increased once again:

[hawkeye217]

Did you try swapping the cable?

[ElectronicBattle]

No - it's the standard USB3.0 cable which came with the Coral. It's certainly something I can try but it seems unlikely to be the culprit (seeing as it's a static installation, no vibration, it gets unplugged very rarely). But still worth swapping, yes.