macOS JPackage Runtime exec crash

[NateIsStalling]

Please provide a brief summary of the bug

On macOS, OpenJDK 21+35 jpackage MacAppImageBuilder implementation uses the same entitlements path when signing the app and embedded executables, resulting in a crash when a packaged Java application attempts to start a new command line process with Runtime.getRuntime().exec(...) or new ProcessBuilder(args).start().

Per Apple's docs on embedding a command-line tool in a sandboxed app:

Adding other entitlements to the tool can cause problems. If the tool immediately crashes with a code signing error when your app runs the tool, check that the tool is signed with just these two entitlements: com.apple.security.app-sandbox and com.apple.security.inherit.

https://developer.apple.com/documentation/xcode/embedding-a-helper-tool-in-a-sandboxed-app

Since the same entitlements path is reused for the packaged app and embedded frameworks in MacAppImageBuilder.signAppBundle, the entitlements for the embedded frameworks may cause problems starting new child processes from embedded jspawnhelper since the app entitlements are generally not just these two entitlements: com.apple.security.app-sandbox and com.apple.security.inherit.

Did you test with the latest update version?

• Yes

Please provide steps to reproduce where possible

1. create java app with the following main:

package org.example;
// ...

public static void main(String[] args) throws IOException {

        // List files in the current directory using the "ls" command
        Process proc = Runtime.getRuntime().exec("ls", new String [0]);

        BufferedReader stdInput = new BufferedReader(new
                InputStreamReader(proc.getInputStream()));

        BufferedReader stdError = new BufferedReader(new
                InputStreamReader(proc.getErrorStream()));
        String s = null;

        // Read the output from the command
        while ((s = stdInput.readLine()) != null) {
            System.out.println(s);
        }

        // Read any errors from the attempted command
        System.out.println("Here is the standard error of the command (if any):\n");
        while ((s = stdError.readLine()) != null) {
            System.out.println(s);
        }
    }

2. build jar
3. run jpackage with mac-sign and mac-app-store args:

jpackage "@jpackage/jpackage.cfg" \
  "@jpackage/jpackage-mac-app-store.cfg" \
  --app-version "$APP_RELEASE_VERSION" \
  --mac-sign \
  --mac-app-store \
  --mac-package-signing-prefix "$MAC_SIGNING_PACKAGE_SIGNING_PREFIX" \
  --mac-signing-key-user-name "$MAC_SIGNING_KEY_USER_NAME" \
  --mac-entitlements sandbox.plist \
  --verbose

5. install dmg output of jpackage on macOS
6. launch installed app - NOTE: macOS gatekeeper may block app from launching without notarization or overriding system settings

related stackoverflow

https://stackoverflow.com/questions/75852613/embedded-command-line-tool-called-via-runtime-exec-in-a-java-sandboxed-app-on-ma

jpackage correction:

I was able to temporarily resolve macOS signing issue by modifying jdk.jpackage.internal.MacAppImageBuilder.signAppBundle() on openjdk tag jdk-21+35:

• Added parameter Path inheritedEntitlements
• Updated first two calls to getCodesignArgs to use inheritedEntitlements instead of entitlements - signing the app itself uses entitlements
• Defined additional plist file for inheritedEntitlements with:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.inherit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
  </dict>
</plist>

NOTE: com.apple.security.cs.allow-unsigned-executable-memory and com.apple.security.cs.disable-library-validation appeared to be required in my case, though Apple only documents com.apple.security.app-sandbox and com.apple.security.inherit as required for embedded command-line tools - having the flexibility to define the embedded command-line tool plist may be required for other users' usecases.

Expected Results

command line tool successfully starts a new process on Mac OS when called with Runtime.getRuntime().exec(...) or new ProcessBuilder(args).start()

Actual Results

jspawnhelper crash on macOS - "Failed to exec spawn helper", "signal 4" appears in console logs

What Java Version are you using?

openjdk 17.0.8.1 2023-08-24 OpenJDK Runtime Environment Temurin-17.0.8.1+1 (build 17.0.8.1+1) OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (build 17.0.8.1+1, mixed mode, sharing)

What is your operating system and platform?

macOS Sequoia 15.3.1 on Intel

How did you install Java?

https://adoptium.net/temurin/releases/?version=17

macOS x64 .pkg

Did it work before?

No

Did you test with other Java versions?

openjdk jdk-21+35 build

Relevant log output

[karianna]

Please try with 21.0.6 and report back.