"Unable to install Python NSLog handler" error without the com.apple.security.cs.disable-library-validation entitlement

[michelcrypt4d4mus]

i'm working on a macos Xcode app and i just ran into an unexpected (to me) situation w/r/t ctypes and the com.apple.security.cs.disable-library-validation entitlement. i don't think it's a bug per se and i also didn't want to clutter up #1861 even more with this question / comment so i figured a "Discussion" made more sense (let me know if i'm mistaken).

so i have successfully built and manually re-signed my app without com.apple.security.cs.disable-library-validation for a few days now. everything had been looking good (and still does)... but then i tried to get the app going with briefcase run and it immediately crashed out with the message Unable to install Python NSLog handler.

some quick testing revealed that in the usual environment - i don't usually run my app with briefcase run - the app still worked fine. a little more digging lead me to the actual stack trace:

Traceback (most recent call last):
  File "/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/app_packages/nslog.py", line 9, in <module>
    import ctypes
  File "/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/support/python-stdlib/ctypes/__init__.py", line 8, in <module>
    from _ctypes import Union, Structure, Array
ImportError: dlopen(/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/support/python-stdlib/lib-dynload/_ctypes.cpython-312-darwin.so, 0x0002): tried: '/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/support/python-stdlib/lib-dynload/_ctypes.cpython-312-darwin.so' (code signature in <798C134E-70FE-3C5F-9515-FF487E944F5C> '/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/support/python-stdlib/lib-dynload/_ctypes.cpython-312-darwin.so' not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)), '/System/Volumes/Preboot/Cryptexes/OS/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/support/python-stdlib/lib-dynload/_ctypes.cpython-312-darwin.so' (no such file), '/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/support/python-stdlib/lib-dynload/_ctypes.cpython-312-darwin.so' (code signature in <798C134E-70FE-3C5F-9515-FF487E944F5C> '/Users/uzor/workspace/my_app/briefcase_my_app/build/my_app/macos/xcode/build/Release/my_app Server.app/Contents/Resources/support/python-stdlib/lib-dynload/_ctypes.cpython-312-darwin.so' not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?))
2024-06-08 22:17:53.802 my_app Server[47312:467105] Unable to install Python NSLog handler

So my conclusion is that the only reason my app still works is because i manually copy the .app produced by briefcase build somewhere else and redo all the code signing manually, which includes re-signing _ctypes.cpython-312-darwin.so with my own certificate... which is fine, but i had a few comments that are almost but not quite questions.

1. I was mildly surprised that the error bubbled up from NSLog. Some comments over on Allow configuration of custom build params for macos Xcode builds in pyproject.toml #1861 made me think that it was really Toga and the GUI system that were the parts of the briefcase suite that needed these entitlements and in fact the reason i even was experimenting with removing them was because my app has no GUI element.
2. Rather than defaulting to a relatively dangerous entitlement like com.apple.security.cs.disable-library-validation as the default build couldn't briefcase build just do what i did and re-sign it so that briefcase run can work? Maybe briefcase package already does this later in the release process and i just don't know about it.

really the only reason i bring it up is because my gut says that Apple will be highly skeptical of approving an app for the app store with the com.apple.security.cs.disable-library-validation entitlement and if it's reasonably straightforward for briefcase build to build apps (even just CLI apps) without it might be worth doing? just my $0.02.

[freakboy3742]

i figured a "Discussion" made more sense (let me know if i'm mistaken).

Exactly the right approach. If it's clearly a bug, log a ticket; if you've got a clearly defined use case for a new feature, open a ticket; if in doubt; open a discussion.

so i have successfully built and manually re-signed my app without com.apple.security.cs.disable-library-validation for a few days now.
...
So my conclusion is that the only reason my app still works is because i manually copy the .app produced by briefcase build somewhere else and redo all the code signing manually, which includes re-signing _ctypes.cpython-312-darwin.so with my own certificate... which is fine, but i had a few comments that are almost but not quite questions.

The error indicates that the ctypes library (which is part of the CPython standard library) either hasn't been signed, has been signed with an ad-hoc certificate, or has been signed with a certificate that is inconsistent with the certificate used to sign the application binary.

If you're doing a straightforward briefcase build/run, then the binary will be signed ad-hoc, which would be consistent with the error you've described. If I'm understanding you correctly, when you turn off the disable-library-validation flag, and fully sign the app on your own, the app works. That would also be consistent, as the app is now fully (and consistently) signed.

An interesting "proof of concept" would be to disable the flag, then running briefcase package -p app. This will prompt you to provide a signing identity, will fully sign your app bundle with that identity, and then zip the result. Unzip that zip file (in the dist folder) and run it; it should succeed (even though briefcase run with the same app will fail). This is because the app is now fully signed (by briefcase), not just using an adhoc identity.

1. I was mildly surprised that the error bubbled up from NSLog.

The reason for this is that the mechanism that Briefcase uses to make stdout/stderr appear in the system log involves using ctypes to bind to the foundation libraries, converting sys.stdout into a writer on NSLog. The package that implements this is std-nslog.

This package isn't mandatory for a Briefcase-packaged app, but it is very useful, especially if you have any debug information going to the console. If you want to silence the warning about missing logging, you can install any library that provides an nslog.py file (even an empty one)

2. Rather than defaulting to a relatively dangerous entitlement like com.apple.security.cs.disable-library-validation as the default build couldn't briefcase build just do what i did and re-sign it so that briefcase run can work? Maybe briefcase package already does this later in the release process and i just don't know about it.

It seems like the issue is that adhoc signing requires the extra option; but it's an unnecessary option once full signing is in place (unless, of course, you're doing something else that is "unsafe" from Apple's perspective). This means:

1. We need to use different Entitlements.plist content when signing with an adhoc identity; and/or
2. We need to expose options to build/run to allow the user to select a signing identity, rather than just using an adhoc identity.

We've deliberately defaulted to adhoc signing for the build/run process because it's much easier for first-time users, so we'd want to preserve that as a default for build/run; but otherwise the mechanics for selecting a signing identity are in place already.

One additional feature that might improve the UX around this would be to be able to save the default signing identity - so briefcase config set-identity (or something like that) would write a file defining the project's default signing identity, similar to the way that you can set git configuration options on a per-project or global basis. That way you could set the default once, then have every subsequent build use that identity. This could also be useful for setting default simulator images.

really the only reason i bring it up is because my gut says that Apple will be highly skeptical of approving an app for the app store with the com.apple.security.cs.disable-library-validation entitlement and if it's reasonably straightforward for briefcase build to build apps (even just CLI apps) without it might be worth doing? just my $0.02.

Like I've said elsewhere, Mac App Store publication is something we'd like to support, but we don't currently test in any capacity. Resolving this sort of issue would seem to be highly desirable (if not absolutely required) for fully supporting the App Store.

[michelcrypt4d4mus]

An interesting "proof of concept" would be to disable the flag, then running briefcase package -p app.

ended up doing this at some point and yeah the app ran just fine.