[Config Support]: Tricky camera settings help (false positives)

[matknowles]

Describe the problem you are having

Hi, I'm having a bit of trouble with false positives on a camera that overlooks my driveway. I've set everything I can think of short of a motion mask, but as the cars dont stay in the same spot I don't see how that'll fix the issue.

It seems like reflections in the car bodywork is being mistaken for a person. Also lights at night with passing cars triggers as a person sometimes too (worth noting not too often, but it's linked to TTS notifications on speakers all around my house so it's a bit annoying when it happens)

I've tested and set the ratio,size and I've kept bumping min_score and threshold up to help, but it's having the opposite effect now. An example, yesterday there was a bit of direct sunlight and it didn't detect actual people on the driveway.

If it matters, I'm using a Coral USB detector. OPENVINO seems to work well on my new machine, but it adds 10-15% more CPU usage constantly, so is not preferable unless a different detection model might fix it.

I seem to recall a discussion where someone said not to have the min_score and threshold too similar, so maybe this is my issue, but I've tried to keep them as low as possible to maximise detections in tricky lighting etc.

I've also submitted 20 images on Frigate+for this camera, so maybe I could submit more?

Thanks for any advice or help, (I've removed the image out of paranoia.)

I think it's unlucky yet hilarious that the reflections in the bodywork match the size and ratio of an actual person as well, typical.

Version

0.15.0-6b12a45

Frigate config file

Driveway:
    webui_url: http://192.168.1.14/#/home
    detect:
      #width: 704
      #height: 576
      fps: 5
    enabled: true
    audio:
      enabled: true
      listen:
        - bark
        - fire_alarm
        - scream
        - speech
        - yell
        - bird
        - chirp
        - dogs
        - car
        - emergency_vehicle
        - aircraft
        - siren
        - power_tool
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Driveway_Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
            - audio
        - path: rtsp://127.0.0.1:8554/Driveway
          input_args: preset-rtsp-restream
          roles:
            - record
    #record:
    #  retain:
    #    days: 0
    #    mode: all
    objects:
      filters:
        person:
          min_area: 8000 #person at the end of the driveway
          max_area: 135000 #person at side/closer to camera
          min_ratio: 0.25
          max_ratio: 0.5
          min_score: 0.81
          threshold: 0.8
          #min_score: 0.15 #for testing
          #threshold: 0.1 #for testing
    zones:
      DrivewayBetweenCars:
        coordinates: 0.578,0.37,0.382,0.369,0.32,1,0.645,1
        objects:
          - person
          - umbrella
        #filters:
        #  person:
        #    #min_area: 8000
        #    #max_area: 135000
        #    threshold: 0.73
        #    min_score: 0.5
        notifications:
          enabled: true
          mqtt_topic: frigate/driveway/DrivewayBetweenCars
        inertia: 3
        loitering_time: 0
      Driveway_General:
        coordinates: 
          0.19,0.079,0.204,0.376,0.822,0.371,0.866,0.4,0.912,0.42,1,0.545,0.998,0.998,0.231,0.999,0.1,0.368,0.004,0.392,-0.001,0.231
        loitering_time: 0
        objects:
          - backpack
          - bicycle
          - bird
          - cat
          - dog
          - eye glasses
          - handbag
          - hat
          - horse
          - package
          - person
          - suitcase
          - umbrella
        inertia: 3
    review:
      alerts:
        labels:
          - person
        required_zones:
          - DrivewayBetweenCars
          - Driveway_General
    motion:
      mask: 0.779,0.917,0.995,0.922,0.995,0.943,0.777,0.941
      threshold: 30
      contour_area: 15
      improve_contrast: 'true'

Relevant Frigate log output

N/A

Relevant go2rtc log output

N/A

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

docker-compose file or Docker CLI command

N/A

Object Detector

Coral

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[hawkeye217]

This is almost certainly model related.

I'd definitely suggest submitting many more examples to Frigate+ and then using a tuned Frigate+ model. I had similar issues with reflections causing false positives, which have all been mitigated now with a Frigate+ model.

[NickM-27]

I'm hoping not hundreds as I have to label every object in each image which is pretty tedious.

To be clear if you subscribe to Frigate+ then your images will be automatically annotated with the suggestions feature

[hawkeye217]

I noticed a difference after perhaps a few dozen images, with an 80/20 ratio of true positives to false positives.

[matknowles]

Yes I am subscribed. Didn't know about that, it's been a little while since I did any submissions because of that very issue, that sounds great, thanks.

[matknowles]

Oh wow that's fantastic. Just checked and indeed everything is automatically boxed. That's my afternoon sorted. I'll try and post back with results.

Do I have to resubmit a new model request to incorporate the new images? I'd assume so.

[hawkeye217]

Do I have to resubmit a new model request to incorporate the new images? I'd assume so.

Yes

[matknowles]

As an update, my plan was to reduce the detection sensitivity, flag false positives in frigate+, get a new model then hope the false positives stop. I've requested a new model and incorporated it into my config, but unlike the initial case above, it seems like (at least some of) the false positives aren't showing in review or explore so that I can submit them as false positives. I'm wondering if it's an issue with how my home assistant automation is setup?

It's pretty simple; if person occupancy becomes occupied in either of the driveway zones it sends a TTS alert to my speakers.

I'm wondering why HA might be getting that trigger but frigate isn't recording it as an event/detection? I'm assuming it's my config, as that's all my tiny brain can come up with.

  Driveway:
    webui_url: http://192.168.1.14/#/home
    #detect:
      #width: 704
      #height: 576
     # fps: 5
    enabled: true
    audio:
      enabled: true
      listen:
        - bark
        - fire_alarm
        - scream
        - speech
        - yell
        - bird
        - chirp
        - dogs
        - car
        - emergency_vehicle
        - aircraft
        - siren
        - power_tool
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Driveway_Third
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Driveway
          input_args: preset-rtsp-restream
          roles:
            - record
            - audio
    #record:
    #  retain:
    #    days: 0
    #    mode: all
    objects:
      filters:
        person:
          min_area: 8000
          max_area: 135000
          min_ratio: 0.25
          max_ratio: 0.5
          min_score: 0.7
          threshold: 0.68
          #min_score: 0.15
          #threshold: 0.1
    zones:
      DrivewayBetweenCars:
        coordinates: 0.578,0.37,0.382,0.369,0.32,1,0.645,1
        objects:
          - person
          - umbrella
        #filters:
        #  person:
        #    #min_area: 8000
        #    #max_area: 135000
        #    threshold: 0.73
        #    min_score: 0.5
        notifications:
          enabled: true
          mqtt_topic: frigate/driveway/DrivewayBetweenCars
        inertia: 3
        loitering_time: 0
      Driveway_General:
        coordinates: 
          0.19,0.079,0.204,0.376,0.822,0.371,0.866,0.4,0.912,0.42,1,0.545,0.998,0.998,0.231,0.999,0.1,0.368,0.004,0.392,-0.001,0.231
        loitering_time: 0
        objects:
          - backpack
          - bicycle
          - bird
          - cat
          - dog
          - eye glasses
          - handbag
          - hat
          - horse
          - package
          - person
          - suitcase
          - umbrella
        inertia: 3
    review:
      alerts:
        labels:
          - person
    motion:
      mask: 0.779,0.917,0.995,0.922,0.995,0.943,0.777,0.941
      threshold: 30
      contour_area: 15
      improve_contrast: 'true'

I can't see how to format the yaml properly, grr!! Sorry about that

Thanks

[hawkeye217]

The HA occupancy sensors don't have filters applied to them, so they may trigger on false positives that don't qualify to be a tracked object in Frigate's UI. You should use the reviews MQTT topic instead, or apply filters to the events topic.

Alternatively, you could use the beta version of the sgtbatten Frigate notifications blueprint: https://github.com/SgtBatten/HA_blueprints

[matknowles]

Oh that might well be it. That's useful to know, thanks very much.

[matknowles]

I've tried for the last couple of hours to get it working, but I'm not knowledgable enough to know what I'm doing wrong.

I tried to use the blueprint like you advised, but couldn't find an option at first to use TTS (media player) as the notification type. I tried settting tts.speak as a custom action (auto trigger), but it didn't notify during a test.

I'm now trying in HA to modify my old automation use mqtt instead of the occupancy sensors and then adding a template to filter/only trigger when either driveway zone is entered (as a person type and severity: alert), but for the life of me I can't get a notification to come through. The template chatGPT (and Claude) gave me is:

condition: - condition: template value_template: > {% set payload = trigger.payload | from_json %} {{ (payload.type in ['update', 'new']) and (payload.camera == 'Driveway') and (payload.severity == 'alert') and ( 'DrivewayBetweenCars' in payload.data.zones or 'Driveway_General' in payload.data.zones ) }}

but it's not working. The template is showing as false during a test.

I saw on a discussion here where someone used separate conditions with each part (label/person/zones etc) as separate conditions, but that's not working for me either. I can see "triggered" flashing as alerts come in to HA and I can see the data in MQTT explorer.

[hawkeye217]

See this tip post: #2898

You'll just need to adapt the service call your automation is making.

[hawkeye217]

You may also find this tip post helpful: #11554

[matknowles]

I've tried using Nick's automation code from May as it was for >0.14 (reviews instead of events), but I have to add conditions to filter out the camera/zones/objects I want notifications for and I think that's what's going wrong. I tested using Nick's automation code as it is combined with a tts action and it worked, but it's announcing all alerts on all cameras (as expected). As I have 11 cameras, most of which are producing alerts pretty often, I only want tts alerts for the front path and driveway basically (except for nighttime, but I'll deal with that after getting notifications reliable). I still want things like cars going past the house or people walking past to be alerts (in frigate), but I only want TTS notifications for say, people crossing onto my path or driveway. The other issue with Nick's code is that it triggers a notification well after the event has ended, so I'm getting weird tts notifications say a minute or 2 after someone's left the camera/zone, which is no good for my needs.

I did try adding conditions to narrow it down:

{{ trigger.payload_json.type in ['new', 'update'] }}
seemed like alerts would come through sporadically at best and still coming through well after the scene is clear. I thought these "well after the scene is clear" events are involved with the 'end' payload, but using this condition doesn't seem to fix them.

I've also tried (for a different camera)

`triggers:

• topic: frigate/reviews
  trigger: mqtt
  conditions:
• condition: template
  value_template: "{{ trigger.payload_json.type in ['new', 'update'] }}"
  alias: Ensure Payload type is 'new' or 'update'
• condition: template
  value_template: "{{ trigger.payload_json.after.camera == 'Garage_Interior' }}"
  alias: Ensure camera is 'Garage_Interior'
• condition: template
  value_template: "{{ trigger.payload_json.after.severity == 'alert' }}"
  alias: Ensure severity is 'alert'
• condition: template
  value_template: "{{ 'person' in trigger.payload_json.after.data.objects }}"
  alias: Ensure objects include 'person'
• condition: template
  value_template: >-
  {% set last_triggered =
  state_attr('automation.garage_interior_frigate_alert',
  'last_triggered') %} {% if last_triggered %} {{ (now() - last_triggered).total_seconds()
  > 90 }} {% else %} true {% endif %}
  alias: Ensure automation hasn't run in the last 90 seconds`

to bit by bit narrow down the alerts, but again it seems sporadic at best and is not working well (I stood on the middle of the driveway for a good 30 seconds, no notification, went inside, no alert in frigate either then 2 minutes later a notification came through as a reflection was detected as a person)

The simplest solution I can think of is if Frigate can be configured to send alerts on a per-camera basis to its own mqtt topic. Is this possible?

[NickM-27]

I don't think there is a reason for a separate topic per camera, you can filter on camera field in the conditions. You can use traces to try and see why this may not be working as you expected.

[matknowles]

I think I may have a deeper problem (with frigate :p). I setup a simple automation using the blueprint for my porch camera, my phone as a device and the custom action tts notification (auto trigger), went into my porch with my phone in hand watching the debug stream with bounding boxes on. I was definitely there outlined as a person, but no notification showed up on my phone or the speakers. The traces show the automation flow starting then jumping to the end, with no notifications triggering.

alias: Porch Frigate Blueprint Alert
description: ""
use_blueprint:
  path: SgtBatten/Beta.yaml
  input:
    presence_filter:
      - ""
    camera:
      - camera.porch
    notify_device: 839c718780932e6315d91556981044ba
    labels:
      - person
    silence_timer: 0
    custom_action_auto:
      - action: tts.speak
        metadata: {}
        data:
          cache: false
          media_player_entity_id: media_player.nest_mini_speakers
          message: Frigate blueprint porch test notification
        target:
          entity_id: tts.piper
    debug: true

Triggered by the mqtt topic frigate/events at 20 December 2024 at 20:00:15 If: then action executed Choose: No action executed Finished at 20 December 2024 at 20:00:15 (runtime: 0.00 seconds)

[NickM-27]

I don't use the blueprint so I can't help with that specifically

[matknowles]

can you tell me if my templates are correct from earlier please? It would help me narrow down whether I have a templating issue or a wider issue with frigate. That would be very helpful if you wouldn't mind.

[NickM-27]

it's difficult to know, there are a lot of templates, I wouldn't expect more than one or two to be needed to accomplish what you are after

[matknowles]

I still can't get it to work satisfactorily, but I have noticed a topic created when a zone becomes active, so for now I'm using its state change from 0 to 1 to trigger the tts notification and initial testing seems to work reasonably well.

alias: DrivewayBetweenCars MQTT Person Alert
triggers:
  - topic: frigate/DrivewayBetweenCars/person/active
    payload: "1"
    trigger: mqtt
actions:
  - target:
      entity_id: tts.piper
    data:
      cache: true
      media_player_entity_id: media_player.nest_mini_speakers
      message: Person detected on the drive
    action: tts.speak

Hopefully I'm most of the way there in fixing the initial issue. I've submitted some more false positive images to +, verified/saved them and requested a new model.

[matknowles]

As an update it seems as if running HAOS in virtualbox may have been at least partially related to the issues. I had my Asus NUC fail (wouldn't power on) and have now moved to a new pc which runs HAOS bare metal and the alerts have been much more stable since. I've yet to try using Nick's more robust notification code for more customised alerts re zones/objects, but it seems a lot better now.