[Bug] ImHex is picked up as 'unresponsive' while using file dialogues

[applecuckoo]

Operating System

Linux

What's the issue you encountered?

When you open a file dialogue, Ubuntu will think that ImHex is unresponsive.

How can the issue be reproduced?

1. Open a file saving dialogue
2. Wait a few seconds
3. You'll see that there is a pop-up on the ImHex main window that says 'ImHex is not responding'

ImHex Version

1.35.3

ImHex Build Type

• Nightly or built from sources

Installation type

.deb file

Additional context?

Tested on Ubuntu 24.04

[WerWolv]

Hey
I'm not sure there's a real solution for this. When the file picker opens, it puts ImHex into a spin-loop so no events are happening in the background anymore and that triggers gnome to think that the app went unresponsive.

Maybe @btzy can answer this better than I can.

[btzy]

Do you mind checking if this bug also occurs before dd60762?

This needs more investigation but my guess is that you're using portals and gnome doesn't realise that the out-of-process dialog is logically part of ImHex.

[WerWolv]

This has always been the case. I just assumed that's how it works when talking to the dbus on Linux

[applecuckoo]

@btzy Yeah, definitely before dd60762 - I usually don't use nightlies and only tried them out specifically for this. I know this isn't intended since this is usually the only time it triggers, even GHex doesn't do this.

[btzy]

I don't currently have Ubuntu 24.04 set up, but I installed the "Ubuntu 22.04 DEB x86_64" package downloaded from GitHub Actions on a mostly clean Ubuntu 22.04 minimal VM and couldn't reproduce the issue even after waiting for a minute. I also don't see any warning pop-up when running NFDe's SDL2 test binary (which can also be downloaded from GitHub Actions if you're unwilling to set up the repo).

Does this only occur on Ubuntu 24.04?

Since GHex is part of gnome, I guess it might use GTK directly instead of portals.

[btzy]

Actually it does occur on Ubuntu 22.04 too, but the warning stays under the file dialog. The warning goes away quickly once control is returned back to ImHex, so it could be simply disappearing fast enough that it is unnoticeable.

On ImHex master, the file dialog stays in the middle of the ImHex window in Ubuntu, which on my system covers any warning popup, making the warning impossible to observe.

[WerWolv]

Definitely happens on Ubuntu 24 as well. And you can move the file picker window around, doing so will show the error. The window also gets a bit darker when the popup shows up

[btzy]

The file dialog stays in the middle on my Ubuntu 22.04 after dd60762 is applied.

I think the real fix here is that the file dialog needs to be presented asynchronously, so that the main loop can still process events. This is a current limitation of NFDe; I think it is possible on all platforms to implement asynchronous dialogs, with varying difficulty. I've looked into implementing asynchronous file dialogs a couple of years back, but I was hesitant to implement it then because it is non-trivial to get the API right. For example, Portal (Linux) requires you to listen on a file descriptor, Cocoa (macOS) requires you to pass a callback, while GTK (Linux) and Windows require you to send (platform-specific) events from the main event loop to the dialog. We'd need to somehow unify all of that in a cross-platform manner (or, bail out and spawn a surrogate process to parent the dialog, which has its own issues).

It appears that both Firefox and Chrome present their file dialogs asynchronously (animations on the main window still continue when the file dialog is open) though.

[applecuckoo]

@btzy just fyi, I got 1.35.4 and now the file dialogue is straight up broken, as in it crashes ImHex.

[btzy]

I don't have access to a computer right now and won't have access for slightly over a week, but I tested it then on Ubuntu and it worked. I haven't gotten any other complaints about this on my repo. Can you perhaps get a backtrace and core dump of the crash? Also, could it possibly be that ImHex is loading the library dynamically, and somehow it's picking v1.1.1 instead of v1.2.0?

[applecuckoo]

I just tested this again and it seems to be particular to the .deb package - the AppImage doesn't have this issue. Annoyingly, it seems to be failing silently, i.e. there isn't the typical crash dialogue the next time you open ImHex. Would there be a way to get this manually?

[WerWolv]

The difference between the AppImage and the .deb package is that the AppImage uses the xdg-desktop-portals backend while the .deb package uses the GTK one. That's probably the reason.
Except for the Fedora packages, all of the releases provided by us link against the bundled version of the library but I can't talk about other distros or packaging systems

[WerWolv]

One more bug report from a user on Wayland + Sway

[btzy]

Thanks for the report.

From the reports here, I suspect that the problem manifests on systems with both X11 and Wayland available, where ImHex opens an X11 main window but GTK opens a Wayland window for the file chooser. An X11 window cannot be made the parent of a Wayland window, and so the crash occurs. This would be a bug in NFDe's GTK backend, so using the portal backend sidesteps the problem.

I'll try to take a closer look this weekend or early next week.

[btzy]

@WerWolv I think btzy/nativefiledialog-extended@ef6afbb should fix the problem, but I can't really test it with ImHex due to #1832.

[btzy]

@applecuckoo do you mind testing if the .deb build from the CI of #1834 fixes the crash? It should fix the crash, and apart from that, the dialog window shouldn't go behind the ImHex window even if you click on the ImHex window while the dialog is open.

[applecuckoo]

@btzy okay... so I got a crash, stacktrace is below. The dialog window now moves with the ImHex window, though I can still see the 'not responding' message when I divide ImHex to the top half of the screen.

20240802_071848.log

[btzy]

Thanks! The crash is not expected of course, but the dialog window moving with the ImHex window is expected on Ubuntu (try opening a file dialog on Firefox or any other "proper" Linux software and the dialog window should move with the main window too).

The "not responding" part is expected for now, and it's much more difficult to solve (I think it requires either multithreading or a proper async file dialog implementation).

When exactly does the crash happen? Since you can see the dialog window, presumably the crash happens after the dialog opens, or when it is closed after you select the file to open?

Do you mind letting me have a look at the core dump if you have such a file?

[applecuckoo]

@btzy I don't know how to set up a core dump, but the error triggers when you drag something within the file dialog. The stacktrace is in my previous comment.