jtop exits without error on high load

[rvandernoort]

Describe the bug

Running a fairly high load on my Nvidia Jetson Nano, while running jtop to get the power measurements. For all options, using docker, python on host, or just jtop in cli, everything crashes at a certain peak usage moment.

To Reproduce

I have quite a complicated setup, so a bit hard to describe here, but what I can say is that GPU usage is like 98% and power draw around 8W when it happens, so I suspect some similar program could reproduce this.

Expected behavior

No crashes, just continued cycle

Additional context

No error messages or exceptions, so if theres any way to get more info about what is going wrong please let me know!

Board

Output from jetson_release -v:

• jetson-stats version: 4.2.4
• P-Number: p3448-0000
• Module: Nvidia Jetson Nano
• Jetpack: 4.6.4
• L4T: 32.7.4

Log from jtop.service

Attach here the output from: journalctl -u jtop.service -n 100 --no-pager

-- Logs begin at Thu 2023-03-02 13:58:02 CET, end at Wed 2024-01-17 12:41:57 CET. --
jan 16 00:20:57 JetsonNano systemd[1]: Started jtop service.
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.service - jetson_stats 4.2.4 - server loaded
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.hardware - Hardware detected aarch64
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.hardware - NVIDIA Jetson 699-level Part Number=699-13448-0000-402 K.0
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.hardware - NVIDIA Jetson Module=NVIDIA Jetson Nano (4 GB ram)
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.hardware - NVIDIA Jetson detected L4T=32.7.4
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.service - Running on Python: 3.6.9
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.cpu - Found 4 CPU
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.gpu - GPU "gpu" status in /sys/devices/57000000.gpu
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.gpu - GPU "gpu" frq in /sys/devices/57000000.gpu/devfreq/57000000.gpu
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.processes - Process service started
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.memory - Found EMC!
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.memory - Memory service started
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.engine - Engines found: [APE NVDEC NVENC NVJPG SE VIC]
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.temperature - Found thermal "PLL" in thermal_zone3
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.temperature - Found thermal "CPU" in thermal_zone1
jan 16 00:20:58 JetsonNano jtop[5373]: [WARNING] jtop.core.temperature - Skipped PMIC
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.temperature - Found thermal "GPU" in thermal_zone2
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.temperature - Found thermal "AO" in thermal_zone0
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.temperature - Found thermal "thermal" in thermal_zone5
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.power - Found I2C power monitor
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.fan - Fan tegra_pwmfan(1) found in /sys/class/hwmon/hwmon1
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.fan - RPM tegra_pwmfan(1) found in /sys/class/hwmon/hwmon1
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.fan - Fan temp controller tegra_pwmfan found in /sys/class/hwmon/hwmon1/temp_control
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.jetson_clocks - jetson_clocks found in /usr/bin/jetson_clocks
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.core.nvpmodel - nvpmodel running in [0]MAXN - Default: 0
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.service - Initialization service
jan 16 00:20:58 JetsonNano jtop[5373]: [INFO] jtop.service - service ready
jan 16 16:35:05 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 16:41:36 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 16:46:45 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 16:53:18 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 16:55:13 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 16:55:20 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 16:56:11 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 16:56:16 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 16:56:16 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 16:56:20 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 16:56:29 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 16:57:08 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 16:57:43 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 17:02:48 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 17:13:38 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 17:22:36 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 17:24:12 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 17:24:29 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 17:26:24 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 17:34:31 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 17:36:07 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 16 17:37:11 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 16 18:02:13 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 17 01:22:52 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close
jan 17 12:24:22 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread started 1000ms
jan 17 12:32:15 JetsonNano jtop[5373]: [INFO] jtop.service - jtop timer thread close

Log from jetson-stats installation

Attach here the output from: sudo -H pip3 install --no-cache-dir -U jetson-stats

Requirement already up-to-date: jetson-stats in /usr/local/lib/python3.6/dist-packages
Requirement already up-to-date: smbus2 in /usr/local/lib/python3.6/dist-packages (from jetson-stats)
Requirement already up-to-date: distro in /usr/local/lib/python3.6/dist-packages (from jetson-stats)