[Support]: "concat" errors somehow affected by home assistant? #13894
Replies: 2 comments 4 replies
-
|
What is your docker compose? |
Beta Was this translation helpful? Give feedback.
-
|
Here's the frigate section. Note I'm using a local macvlan but can't see how this can be relevant. The FRIGATE volume (in caps) is on a separate TrueNAS box. =========================== |
Beta Was this translation helpful? Give feedback.
-
|
basically just seems like something in HA is requesting a clip for a time where there are no recordings saved |
Beta Was this translation helpful? Give feedback.
-
|
I guess that must be somehow related to the Frigate Notifications Blueprint. Is there any way to tell from those logs what recording time it is looking for? I'm wondering if the issue might be a performance thing. Somehow frigate is taking too long to store the clip? If HA is requesting data is it supposed to come from the tmpfs or from the /media folder? The reason I ask is that my /media folder is on an external TrueNAS box and is presumably quite a bit slower than the cached data. Is it worth moving /media onto local storage in the docker host as a test? I'd need to cull some data to make it fit but ready to give it a go if you think there is any point. I indicated before that I have been having issues with my coral accelerator. If installed I do get much faster detection and reduced CPU. It seems to work well except that it regularly crashes with "Detection appears to be stuck. Restarting detection process...". It usually recovers from this but occasionally I've needed a complete restart to recover. However, I've just noticed that when the coral is installed I no longer seem to get the concat / HA errors. This again suggests some sort of performance/timing issue. I've been treating these 2 types of errors as separate problems, but maybe not. Perhaps its all related somehow? |
Beta Was this translation helpful? Give feedback.
-
|
You might want to try a different cable / different port. The included cable is known to have issues with the coral. |
Beta Was this translation helpful? Give feedback.
-
|
I have made a bit of progress. First with the "No files to concat" error. This seems to be a timing problem. As far as I can see When frigate identifies an alert it signals HA via mqtt. I'm using the HA Frigate Notifications (0.12.0.4d) blueprint to send notifications to my cell phone. The notifications are supposed to contain clips and/or snapshot images from the alert, but I noticed that these are often missing and that seems to time-match the errors. Presumably HA has to ask frigate for them and the problem arises when they are not (yet) available. I guessed that maybe the clip recording had not finished maybe because of the event's pre and post_capture settings? So I've turned them down to 2 seconds each, and now everything seems to work fine. I have had no errors for > 2 days compared to every hour or so before. Second with the usb coral accelerator. I have seen lots of discussions online by people having similar problems. I've tried the USB cable, tried different usb ports, the suggested grub "quirks" parameters etc. Everything I could find to no avail. Then a couple of days ago I moved frigate's /media folder away from its NAS NFS share and onto local storage in the docker host. I have had no failures since! No explanation either! The NAS is a TrueNAS SCALE box and it was way back-level so I have now upgraded it to the latest stable release and have moved /media back. So far so good, it has run for a few hours with no coral errors which is probably longer than ever before. Too early to declare victory, but looking promising. The NAS is shared by lots of other apps so it's hard to imagine what could be wrong there, but so far that's the way it's looking. |
Beta Was this translation helpful? Give feedback.
-
Describe the problem you are having
I am a new frigate user so there is a fair chance my problem is self-inflicted. I apologise if I have missed something basic. I have set up a test environment to experiment with a single dahua 5442 camera. I am using 3x docker containers (Frigate, mosquitto and Home Assistant) on the same docker host which is Ubuntu on an intel nuc.
I have some issues with the Coral so I have removed it for now. Without the Coral Frigate appears to be working fine and communicating successfully with home assistant. Except that I am seeing lots of errors in the frigate log.
See the 1st log extract attached. This shows a clean restart of frigate (with my USB Coral removed) and then you can see several examples of my problem. Starting at timestamp 11:13:35 you can see a pattern of 3 log lines. In practice this pattern of 3 lines coincides with frigate detecting something that it deems to be an alert. It is intermittent, not every alert causes the error. It does not occur for a detection that doesn't qualify as an alert.
Notice the timestamp in the 3rd line of the pattern is odd. In practice it matches the time of the most recent alert and frigate is showing this same time for every error.
The text of the error log suggests some sort of data loss, but I can not find anything missing.
I said that the pattern of 3 log lines occurs at the same time as an alert. However, that is only true if the home assistant container is running. If I stop the HA container then frigate no longer puts out the errors but it does continue to update the 3rd line of each group with the latest timestamp every time there is an alert.
In the log extract I stopped the HA container at 16:00 and then caused an alert to be detected at 16:02. You can see that the 3rd line in my 3-line pattern has been updated with that 16:02 timestamp but the 3-line pattern for an alert at 16:02 is not there.
So the actual ERROR messages only occur if HA is running which seems pretty bizarre to me!
Otherwise communication from frigate to HA seems to be working fine. I can see live and review videos inside HA, and I am using HA's frigate notification template to send notifications to HA on my mobile phone.
I have tried turning on debug for the logger but am quickly overwhelmed with data. I have attached an extract for a few minutes around an example error at timestamp 08:12:51.
Any suggestions?
And some thanks: I am astonished an the completeness of frigate. It is an amazing piece of work, congratulations, and many thanks!
===============================================================
Some more info on my environment in case it's relevant:
frigate 0.14.1-f4f3cfa
mosquitto 2.0.18
home assistant 2024.7.4
The docker host is an Intel nuc NUC11TNHi5 with 16GB Ram and a 500GB NVMe M.2 Solid State Drive
The nuc is running ubuntu server 20.04 with kernel 5.15.0-119-generic
The nuc has a bunch of other unrelated containers but is very lightly loaded.
I have a USB Coral accelerator (currently removed) and a TrueNAS server used for frigate storage.
I am experiencing 2x problems with frigate. Problem 1 is related to the Coral so I have removed it and am concentrating the problem above
Version
0.14.1-f4f3cfa
What browser(s) are you using?
Google Chrome 128.0.6613.137
Frigate config file
Relevant Frigate log output
Relevant go2rtc log output
FFprobe output from your camera
Frigate stats
{ "cameras": { "front_garden": { "audio_dBFS": 0, "audio_rms": 0, "camera_fps": 5.1, "capture_pid": 429, "detection_enabled": true, "detection_fps": 0.1, "ffmpeg_pid": 441, "pid": 425, "process_fps": 5.1, "skipped_fps": 0 } }, "cpu_usages": { "1": { "cmdline": "/package/admin/s6/command/s6-svscan -d4 -- /run/service", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "106": { "cmdline": "bash ./run.user go2rtc-healthcheck", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "107": { "cmdline": "python3 -u -m frigate", "cpu": "1.2", "cpu_average": "1", "mem": "2.2" }, "117": { "cmdline": "nginx: master process nginx", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "15": { "cmdline": "s6-supervise s6-linux-init-shutdownd", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "16": { "cmdline": "/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "162": { "cmdline": "nginx: worker process", "cpu": "0.1", "cpu_average": "0", "mem": "0.0" }, "163": { "cmdline": "nginx: worker process", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "164": { "cmdline": "nginx: worker process", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "165": { "cmdline": "nginx: worker process", "cpu": "0.0", "cpu_average": "0", "mem": "0.2" }, "166": { "cmdline": "nginx: cache manager process", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "24": { "cmdline": "s6-supervise nginx-log", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "25": { "cmdline": "s6-supervise frigate-log", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "26": { "cmdline": "s6-supervise certsync-log", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "27": { "cmdline": "s6-supervise go2rtc", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "28": { "cmdline": "s6-supervise frigate", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "29": { "cmdline": "s6-supervise certsync", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "30": { "cmdline": "s6-supervise nginx", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "31": { "cmdline": "s6-supervise s6rc-fdholder", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "32": { "cmdline": "s6-supervise s6rc-oneshot-runner", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "325": { "cmdline": "bash ./run.user certsync", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "33": { "cmdline": "s6-supervise go2rtc-log", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "34": { "cmdline": "s6-supervise go2rtc-healthcheck", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "351": { "cmdline": "frigate.logger ", "cpu": "0.0", "cpu_average": "0", "mem": "0.5" }, "380": { "cmdline": "frigate.recording_manager", "cpu": "0.4", "cpu_average": "0", "mem": "1.0" }, "382": { "cmdline": "frigate.review_segment_manager", "cpu": "0.2", "cpu_average": "0", "mem": "0.7" }, "408": { "cmdline": "/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(57)", "cpu": "0.1", "cpu_average": "0", "mem": "0.1" }, "409": { "cmdline": "frigate.detector.cpu ", "cpu": "1.5", "cpu_average": "26", "mem": "0.7" }, "411": { "cmdline": "frigate.output ", "cpu": "0.4", "cpu_average": "0", "mem": "0.7" }, "425": { "cmdline": "frigate.process:front_garden", "cpu": "1.8", "cpu_average": "2", "mem": "0.8" }, "429": { "cmdline": "frigate.capture:front_garden", "cpu": "0.7", "cpu_average": "0", "mem": "0.7" }, "441": { "cmdline": "ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.14.1-f4f3cfa -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/front_garden_sub -r 5 -vf fps=5,scale_vaapi=w=704:h=576:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:", "cpu": "1.4", "cpu_average": "1", "mem": "0.2" }, "444": { "cmdline": "ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.14.1-f4f3cfa -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/front_garden -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/front_garden@%Y%m%d%H%M%S%z.mp4", "cpu": "0.4", "cpu_average": "0", "mem": "0.1" }, "45": { "cmdline": "/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "459": { "cmdline": "ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://127.0.0.1:8554/front_garden?audio -c:a aac -vn -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/03a3a296b9980d73c1b2da094e2c6ff4", "cpu": "2.0", "cpu_average": "1", "mem": "0.1" }, "46": { "cmdline": "/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "468": { "cmdline": "ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://127.0.0.1:8554/front_garden_sub?audio -c:a aac -vn -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/cc9329de5a1f373cff8d552d7629d985", "cpu": "1.9", "cpu_average": "1", "mem": "0.1" }, "551": { "cmdline": "ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 704x576 -i pipe: -threads 1 -f mpegts -s 880x720 -codec:v mpeg1video -q 8 -bf 0 pipe:", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "556": { "cmdline": "ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "73766": { "cmdline": "sh", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "75641": { "cmdline": "sleep 30s", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "75693": { "cmdline": "sleep 60", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "87": { "cmdline": "s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "88": { "cmdline": "s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "89": { "cmdline": "s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/certsync", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "90": { "cmdline": "s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "913": { "cmdline": "/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(46)", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "95": { "cmdline": "/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml", "cpu": "2.4", "cpu_average": "2", "mem": "0.1" }, "frigate.full_system": { "cpu": "1.9", "mem": "19.5" } }, "detection_fps": 0.1, "detectors": { "cpu": { "detection_start": 0, "inference_speed": 36.48, "pid": 409 } }, "gpu_usages": { "intel-vaapi": { "gpu": "0.04%", "mem": "-%" } }, "processes": { "go2rtc": { "pid": 95 }, "logger": { "pid": 351 }, "recording": { "pid": 380 }, "review_segment": { "pid": 382 } }, "service": { "last_updated": 1726988354, "latest_version": "0.14.1", "storage": { "/dev/shm": { "free": 62.4, "mount_type": "tmpfs", "total": 64, "used": 1.6 }, "/media/frigate/clips": { "free": 1424330, "mount_type": "nfs4", "total": 1679594, "used": 255264 }, "/media/frigate/recordings": { "free": 1424330, "mount_type": "nfs4", "total": 1679594, "used": 255264 }, "/tmp/cache": { "free": 943.7, "mount_type": "tmpfs", "total": 953.7, "used": 10 } }, "temperatures": { }, "uptime": 32770, "version": "0.14.1-f4f3cfa" } }Operating system
Other Linux
Install method
Docker Compose
Object Detector
CPU (no coral)
Network connection
Wired
Camera make and model
Dahua HDW5442TM
Screenshots of the Frigate UI's System metrics pages
No response
Any other information that may be helpful
No response
Beta Was this translation helpful? Give feedback.
All reactions