Problems with insufficient memory in bonsai with pylon camera

[AnnaStuckert]

Hi,

I am running the attached bonsai workflow, which connects to Godot via zeroMQ so when signals are sent from my video game in Godot to Bonsai, this triggers 1) activation of 3 custom built electronics via an Arduino-like (but custom) board, and 2) 5 second video snippets. During my experiment I am continuously recording a video from my Basler pylon camera https://www.baslerweb.com/en/shop/a2a1920-160umpro/ using a pylon capture module in bonsai. This video is both being written into one long video at the end of the experiment, as well as those 5 second video files described.

When running the experiment, I get the following error messages (images attached - for the error on guard stack it also completely shut down bonsai, not just stopped working) - both seems to be related to memory insufficiency. I am using a 32 GB ram Lenovo Thinkpad with over 200 GB available storage, so it would be surprising to me that memory is an issue.

Has anyone had a similar problem or has an idea? the error also occurs if I am only trying to write the 5 second videos and disable the long video writing (but also the error occurs before the end of the experiment, so before the videostream is effectively being written to a file).

When I run the experiment without the camera modules the experiment runs fine and doesn’t start lagging at any point (which it otherwise does for some periods of time)

Any ideas what could be the issue is much appreciated.

Best, Anna

[bruno-f-cruz]

Can you share the workflow too? thanks

[AnnaStuckert]

VR_Godot_basler_FullVideo_5sVideos.bonsai.zip

[bruno-f-cruz]

A few questions:

• What is the resolution, pixel format and fps of your camera?
• How many videos are you concurrently recording via the SelectMany workflow? I can't tell since your TakeUntil is bound to be 5 seconds, but your SelectMany trigger CAN occur every 2 seconds (via the Throttle). There is the potential to accumulate VideoWriters as the experiment progresses.

[AnnaStuckert]

resolution 1936 x 1216
frame rate 100 in the (camera can go up to 160 fps I believe) - attached is the config file for reference
a2A1920-160umPRO_40457504.zip

I am not entirely sure what is meant by pixel format, but pixel size is 3.45 μm × 3.45 μm and the camera is 2.3 MP resolution.

I have currently not set a cap to how many videos can maximally be obtained - but if setting throttle to 5 seconds (so same duration as the written video clip), would that circumvent the risk of accumulating video writers?

[AnnaStuckert]

Hi again, an update on this:
Note: when the memory issue appears, normally the experiment "video game" starts lagging beforehand, usually around 3-4 minutes into the workflow, with memory issues usually presenting at between 10 and 14 minutes.

1. running the experiment at 30 fps instead of 100 fps resulted in lagging at 3 mins and memory error at 3 minutes
2. running the workflow disabling the 5s video snippet writer but keeping the complete video writer enabled resulted in memory issue at 11 minutes lagging at 3 minutes.
3. running the workflow disabling the full video writer but enabling the 5s video writer it runs for the full experiment duration (15 minutes) - maybe I had not properly restarted the experiment yesterday, so possibly the memory issue I encountered yesterday in this case was a spillover effect from a previous run.
4. running the workflow disabling both video writers but keeping the pylon capture enabled, it runs for the full experiment duration (15 minutes)
5. running the workflow with both video writers but setting throttle for the select many to 6 seconds instead of 2, started lagging at 3 minutes and memory error at 10.45 min.

It seems like there is some kind of memory insufficiency. It would be really nice to have the complete video as well, in addition to the shorter videos, but setting the fps lower did not seem to fix it.

[bruno-f-cruz]

That's quite a large video resolution. By pixel format I mean whether you are doing rgb/gray and 8/16bit.
Depending on your machine I am not super surprised you can't keep up with encoding. Can you send us the computer specs? Given that you don't seem to have problems when simply acquiring the video I am tempted to say that it is not a memory leak problem upstream.

[AnnaStuckert]

I see, the format is mono8

This is the specs copied from system overview - please let me know if any other info would be useful

OS Name Microsoft Windows 11 Enterprise
Version 10.0.22631 Build 22631
Other OS Description Not Available
OS Manufacturer Microsoft Corporation
System Name PC29445
System Manufacturer LENOVO
System Model 20W1S4S301
System Type x64-based PC
System SKU LENOVO_MT_20W1_BU_Think_FM_ThinkPad T14 Gen 2i
Processor 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz, 2995 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date LENOVO N34ET59W (1.59 ), 07/11/2023
SMBIOS Version 3.2
Embedded Controller Version 1.44
BIOS Mode UEFI
BaseBoard Manufacturer LENOVO
BaseBoard Product 20W1S4S301
BaseBoard Version SDK0R32862 WIN
Platform Role Mobile
Secure Boot State On
PCR7 Configuration Elevation Required to View
Windows Directory C:\WINDOWS
System Directory C:\WINDOWS\system32
Boot Device \Device\HarddiskVolume1
Locale United Kingdom
Hardware Abstraction Layer Version = "10.0.22621.2506"
User Name ST-ANDREWS\avs20
Time Zone GMT Standard Time
Installed Physical Memory (RAM) 32.0 GB
Total Physical Memory 31.7 GB
Available Physical Memory 24.2 GB
Total Virtual Memory 91.2 GB
Available Virtual Memory 80.3 GB
Page File Space 59.5 GB
Page File C:\pagefile.sys
Kernel DMA Protection On
Virtualization-based security Running
Virtualization-based security Required Security Properties Base Virtualization Support, Secure Boot
Virtualization-based security Available Security Properties Base Virtualization Support, Secure Boot, DMA Protection, UEFI Code Readonly, SMM Security Mitigations 1.0, Mode Based Execution Control, APIC Virtualization
Virtualization-based security Services Configured Credential Guard, Hypervisor enforced Code Integrity, Secure Launch, SMM Firmware Measurement
Virtualization-based security Services Running Credential Guard, Hypervisor enforced Code Integrity, Secure Launch, SMM Firmware Measurement
Windows Defender Application Control policy Enforced
Windows Defender Application Control user mode policy Off
Device Encryption Support Elevation Required to View
A hypervisor has been detected. Features required for Hyper-V will not be displayed.