Failed to make bytecode for arm64 on Linux x86

[robvanderleek]

What version of pkg are you using?

5.15.0

What version of Node.js are you using?

21.2.0

What operating system are you using?

Ubuntu

What CPU architecture are you using?

x86_64

What Node versions, OSs and CPU architectures are you building for?

node20-linuxstatic-arm64

Describe the Bug

Building a static binary for Linux ARM64 failes on Linux x86. For example, this command:

npx pkg -o index -t node20-linuxstatic-arm64 index.js

Generates the following error:

> [email protected]
> Warning Failed to make bytecode node20-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

Adding the --debug flag to the build shows:

/home/rob/.pkg-cache/v3.5/fetched-v20.17.0-linuxstatic-arm64: 1: Syntax error: "(" unexpected
/home/rob/.pkg-cache/v3.5/fetched-v20.17.0-linuxstatic-arm64: 1: Syntax error: "(" unexpected
> Warning Failed to make bytecode node20-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

I am able to generate an ARM64 static Linux binary on macOS.

Expected Behavior

A static Linux ARM64 is generated without warnings.

To Reproduce

I created a minimal example repository that demonstrates the problem:

git clone [email protected]:robvanderleek/yao-pkg-linuxstatic-arm64-bug.git

cd yao-pkg-linuxstatic-arm64-bug/
npm install
npx pkg -o index -t node20-linuxstatic-arm64 index.js

Not that:

npx pkg -o index -t node20-linuxstatic-x64 index.js

does not generate the warning.

[robertsLando]

@robvanderleek Could you tell me if it's woking with past versions of pkg or other nodejs versions like 18/16? I just want to understand where the issue could be

It should come from:

pkg/lib/fabricator.ts

Line 91 in db7f2c5

`Failed to make bytecode ${fabricator.nodeRange}-${fabricator.arch} for file ${snap} error (${error.message})`,

Could you debug it?

[robvanderleek]

Hi @robertsLando

See below how it fails for ARM64 with nodejs 20/18/16 versions, but works for x86 with nodejs 20/18/16.

I've tried pkg versions 5.15.0, 5.12.0, and 5.10.0, all failing.

I think it's host platform related, since building for arm64 fails on Linux Ubuntu 24, but succeeds on macOS.

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node20-linuxstatic-arm64 index.js
> [email protected]
(node:19081) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v20.17.0-linuxstatic-arm64  [====================] 100%

> Warning Failed to make bytecode node20-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node18-linuxstatic-arm64 index.js
> [email protected]
(node:19116) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v18.20.4-linuxstatic-arm64  [====================] 100%

> Warning Failed to make bytecode node18-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node16-linuxstatic-arm64 index.js
> [email protected]
(node:19151) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v16.20.2-linuxstatic-arm64  [====================] 100%

> Warning Failed to make bytecode node16-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js error (write EPIPE)

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node20-linuxstatic-x64 index.js
> [email protected]
(node:19186) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v20.17.0-linuxstatic-x64    [====================] 100%

  fetched-v20.17.0-linux-x64          [====================] 100%

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node18-linuxstatic-x64 index.js
> [email protected]
(node:19224) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v18.20.4-linuxstatic-x64    [====================] 100%

  fetched-v18.20.4-linux-x64          [====================] 100%

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node16-linuxstatic-x64 index.js
> [email protected]
(node:19261) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v16.20.2-linuxstatic-x64    [====================] 100%

  fetched-v16.20.2-linux-x64          [====================] 100%

[robertsLando]

@robvanderleek I actually have no time to look at this issue but I suggest to try debug it to see if you are able to find the root cause of the issue, aboce I linked you the code that makes the bytecode, try to put some breakpoints there to see what's the line that triggers that issue

[thisjt]

I'm able to replicate the issue in WSL (Ubuntu). This might be useful, I think.

I still have no idea where this Syntax error: "(" unexpected is coming from, as on my debug I think this is run asynchronously somewhere.

[robertsLando]

That comes from fabricator fabricate method that spaws the script you find on top to create the bytecode of the file

[jcs090218]

I've same issue in https://github.com/emacs-eask/cli/actions/runs/12576251914/job/35052168042?pr=292#step:5:24.

> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/eask
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/clean/all.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/clean/autoloads.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/clean/dist.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/clean/elc.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/clean/log-file.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/clean/pkg-file.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/clean/workspace.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/analyze.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/archives.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/bump.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/cat.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/clean.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/compile.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/concat.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/create.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/docker.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/docs.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/emacs.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/eval.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/exec-path.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/exec.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/files.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/format.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/generate.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/info.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/init.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/install-deps.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/install.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/keywords.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/link.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/lint.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/core/list.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/test/ert-runner.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/test/ert.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/test/melpazoid.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/util/locate.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/util/upgrade-eask.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/test/buttercup.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/test/ecukes.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/test/ert-runner.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/test/ert.js error (write EPIPE)
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/workflow/circle-ci.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/workflow/github.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/workflow/gitlab.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/cmds/generate/workflow/travis-ci.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/src/env.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/src/util.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/which/lib/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/yargs/index.cjs
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/isexe/dist/cjs/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/yargs/build/index.cjs
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/cliui/build/index.cjs
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/escalade/dist/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/get-caller-file/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/require-directory/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/string-width/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/y18n/build/index.cjs
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/yargs-parser/build/index.cjs
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/isexe/dist/cjs/options.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/isexe/dist/cjs/posix.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/isexe/dist/cjs/win[32](https://github.com/emacs-eask/cli/actions/runs/12576251914/job/35052168042?pr=292#step:5:33).js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/strip-ansi/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/wrap-ansi/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/escalade/sync/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/emoji-regex/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/is-fullwidth-code-point/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/ansi-regex/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/ansi-styles/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/color-convert/index.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/color-convert/conversions.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/color-convert/route.js
> Warning Failed to make bytecode node18-arm64 for file /snapshot/cli/node_modules/color-name/index.js
> Warning Unable to sign the macOS executable
  Due to the mandatory code signing requirement, before the
  executable is distributed to end users, it must be signed.
  Otherwise, it will be immediately killed by kernel on launch.
  An ad-hoc signature is sufficient.
  To do that, run pkg on a Mac, or transfer the executable to a Mac
  and run "codesign --sign - <executable>", or (if you use Linux)
  install "ldid" utility to PATH and then run pkg again

[jcs090218]

> @emacs-eask/[email protected] pkg-linux-arm64
> pkg package.json -t node20-linux-arm64 --output dist/eask

> [email protected]
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v20.18.1-linux-arm64        [====================] 100%

node:internal/child_process:421
    throw new ErrnoException(err, 'spawn');
    ^

Error: spawn UNKNOWN
    at ChildProcess.spawn (node:internal/child_process:421:11)
    at spawn (node:child_process:761:9)
    at fabricate (D:\_workspace\_eask\cli\node_modules\@yao-pkg\pkg\lib-es5\fabricator.js:55:51)
    at fabricateTwice (D:\_workspace\_eask\cli\node_modules\@yao-pkg\pkg\lib-es5\fabricator.js:120:5)
    at MultiStream._queue (D:\_workspace\_eask\cli\node_modules\@yao-pkg\pkg\lib-es5\producer.js:306:68)
    at MultiStream._next (D:\_workspace\_eask\cli\node_modules\multistream\index.js:86:12)
    at Meter.onEnd (D:\_workspace\_eask\cli\node_modules\multistream\index.js:129:12)
    at Object.onceWrapper (node:events:633:28)
    at Meter.emit (node:events:519:28)
    at endReadableNT (node:internal/streams/readable:1696:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -4094,
  code: 'UNKNOWN',
  syscall: 'spawn'
}

[robertsLando]

@jcs090218 is this happening only with node20? Could you try to use 18-16?

[jcs090218]

@jcs090218 is this happening only with node20? Could you try to use 18-16?

Here is the latest result; see https://github.com/emacs-eask/cli/actions/runs/12580756239/job/35063275367#step:5:18.

I've tried node18, node20 (I think this is the default to GitHub Actions) and node*.

[robertsLando]

@jcs090218 could it be those modules are esm?

[jcs090218]

@jcs090218 could it be those modules are esm?

No, it's only CJS. Here is the package.json file: https://github.com/emacs-eask/cli/blob/master/package.json.

I can build for x86, but not for ARM. The results are the same across all operating systems: Linux, macOS, and Windows. If you take a closer look to the link I have pasted in the previous comment: https://github.com/emacs-eask/cli/actions/runs/12580756239.

All arm builds are failing.

[robertsLando]

@faulpeltz any clue?

[faulpeltz]

Do we have a small sample where this can be reproduced? does this also happen when building from on linux x64 -> linux static x64 ?

[faulpeltz]

Also, make sure for that the qemu-user-binfmt pacakge is installed for building cross-arch, because the fabricator needs to run the native binary

[jcs090218]

Also, make sure for that the qemu-user-binfmt pacakge is installed for building cross-arch, because the fabricator needs to run the native binary

Got it. That was the problem. I got all warnings solved but the executable no longer works...

You can download the executable at https://github.com/emacs-eask/cli/actions/runs/12660523277 and try it out. Only Linux is working.

[faulpeltz]

which node version are you using for the pkg binaries?
if you don't mind including the sources in the binaries, then you can build with --public that might help

[jcs090218]

if you don't mind including the sources in the binaries, then you can build with --public that might help

The --public seems to be working fine. I wonder what does it do? 🤔 I've read the README but still quite confusing.

   --public             speed up and disclose the sources of top-level project

[faulpeltz]

The --public seems to be working fine. I wonder what does it do? 🤔 I've read the README but still quite confusing.

It includes the script source for the project in the result binary instead of just the code cache bytecode. This is a little bit slower because of additional steps on startup for parsing/compiling, and is basically the same as running node main.js but everything bundled in the snapshot file system.

Getting code-cache-only to work is currently an open issue in Node 22, but should work with earlier versions.