Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[prebuild] feat(prebuild): support of Alpine binaries #2370

Open
wants to merge 7 commits into
base: prebuilds
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
417 changes: 185 additions & 232 deletions .github/workflows/prebuild.yaml

Large diffs are not rendered by default.

9 changes: 1 addition & 8 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,14 +1,7 @@
build
.DS_Store
.lock-wscript
test/images/*.png
examples/*.png
examples/*.jpg
testing
out.png
out.pdf
out.svg
.pomo

node_modules
package-lock.json

Expand Down
3 changes: 3 additions & 0 deletions prebuild/Alpine/preinstall.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env sh

apk --no-cache add build-base cairo-dev jpeg-dev pango-dev giflib-dev librsvg-dev pixman-dev patchelf
3 changes: 3 additions & 0 deletions prebuild/Alpine/uninstall.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env sh

apk --purge del build-base cairo* jpeg* pango* giflib* librsvg* pixman*
9 changes: 9 additions & 0 deletions prebuild/Debian/bundle.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/usr/bin/env sh

TARGET=./source/build/Release

for lib in $(ldd "${TARGET}/canvas.node" | grep '=>' | cut -d " " -f 3); do
echo "Copy ${lib}"
cp -L "${lib}" "${TARGET}"
patchelf --force-rpath --set-rpath '$ORIGIN' "${TARGET}/$(basename -- "${lib}")"
done
4 changes: 4 additions & 0 deletions prebuild/Debian/preinstall.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/usr/bin/env sh

sudo apt-get update
sudo apt-get install -y build-essential libcairo2-dev libjpeg-dev libpango1.0-dev libgif-dev librsvg2-dev libpixman-1-dev patchelf
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chearon did you build all of the Linux deps from source for a reason, or can we use precompiled packages as this PR does?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need to build on an old OS for maximum compatibility. Programs built against a newer libc may have symbols that don't exist on older OSes (this happened to real users). But we want newer libraries, so have to build from source.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we want newer libraries

👍 forgot about this part.

Copy link
Author

@Delagen Delagen Jun 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You describe preference to run code on old OS, but you require at least NodeJS 18.

Node 18 via NVM ubuntu 14.04

node: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by node)
node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by node)
node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by node)
node: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by node)
node: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)
node: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by node)

Node 18 via NVM Ubuntu 18.04

node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by node)

Node-source distribution doesn't target any Ubuntu lower than 20.04
https://github.com/nodesource/distributions?tab=readme-ov-file#ubuntu-versions

Current build with not precompiled libraries works perfectly on Ubuntu 20.04 with Node 18 from NVM.

Npm test result

 206 passing (125ms)
  6 pending
  3 failing

  1) Canvas
       color parser:

      AssertionError [ERR_ASSERTION]: 'rgba(255, 200, 90, 0.40)' == '#ffc85a'
      + expected - actual

      -rgba(255, 200, 90, 0.40)
      +#ffc85a

      at Context.<anonymous> (test/canvas.test.js:217:12)
      at process.processImmediate (node:internal/timers:476:21)

  2) Canvas
       Context2d#measureText()
         works:

      AssertionError [ERR_ASSERTION]: Expected 14 to be 0.5 +/- 0.5
      + expected - actual

      -false
      +true

      at assertApprox (test/canvas.test.js:24:3)
      at Context.<anonymous> (test/canvas.test.js:1029:7)
      at process.processImmediate (node:internal/timers:476:21)

  3) Canvas
       Context2d#measureText()
         actualBoundingBox is correct for left, center and right alignment (#1909):

      AssertionError [ERR_ASSERTION]: Expected 39 to be 21 +/- 6
      + expected - actual

      -false
      +true

      at assertApprox (test/canvas.test.js:24:3)
      at Context.<anonymous> (test/canvas.test.js:1057:7)
      at process.processImmediate (node:internal/timers:476:21)

That pretty the same issues as Alpine build.

Any other suggestions to build from source?

I thinks that build from source was designed as a way to bundle libraries with compiled node binding, because base libraries without patching tries to resolve dependencies via precompiled path. Patching libraries with patchelf resolves this issue. So I don't think that manually compile libraries is overkill.

3 changes: 3 additions & 0 deletions prebuild/Debian/uninstall.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# FIXME: error if removing with build-essential libcairo2* libjpeg* libpango1.0* libgif* librsvg2* libpixman-1*
sudo apt-get purge -y build-essential libcairo2-dev libjpeg-dev libpango1.0-dev libgif-dev librsvg2-dev libpixman-1-dev
sudo apt-get autoremove --purge -y
43 changes: 0 additions & 43 deletions prebuild/Linux/Dockerfile

This file was deleted.

55 changes: 0 additions & 55 deletions prebuild/Linux/binding.gyp

This file was deleted.

11 changes: 0 additions & 11 deletions prebuild/Linux/bundle.sh

This file was deleted.

10 changes: 0 additions & 10 deletions prebuild/Linux/preinstall.sh

This file was deleted.

84 changes: 0 additions & 84 deletions prebuild/Windows/binding.gyp

This file was deleted.

69 changes: 69 additions & 0 deletions prebuild/Windows/bundle.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#!/usr/bin/env sh

LIBS=()

WINDIR=$(cygpath -u -W)

TARGET=./source/build/Release

# Can be totally replaced with this, but MSYS throw format error if DLL has not .dll extension
#for lib in $(ldd "${TARGET}/canvas.node" | grep '=>' | cut -d " " -f 3); do
# if [[ "${lib}" == "${WINDIR}"* ]]; then
# :
# else
# echo "copy ${lib} to destination"
# cp "${lib}" "${TARGET}"
# fi
#done

shopt -s nocasematch
function addToLibs() {
local lib=${1} && shift
if [[ "$(which "${lib}")" == "${WINDIR}"* ]]; then
:
else
for libItem in ${LIBS[@]}; do
if [[ "${libItem}" == "${lib}" ]]; then
return
fi
done
echo $lib
fi
}

RECURSE_INDEX=0

function getLibsForBinary() {
local binary=$1 && shift

local -i count=0
for binLib in $(objdump -p "$(cygpath -u "${binary}")" | grep "DLL Name:" | sed -e 's/^\s*DLL\sName:\s*//'); do
if [[ ! -z "$(addToLibs ${binLib})" ]]; then
echo "added ${binLib}"
count=$count+1
LIBS+=("${binLib}")
fi
done

if [[ count -gt 0 ]]; then
local currentIndex=${RECURSE_INDEX}
RECURSE_INDEX=${#LIBS[@]}
# recurse if any added from last checked
echo "recurse check after ${count} added"
for lib in ${LIBS[@]:${currentIndex}}; do
getLibsForBinary "$(which "${lib}")"
done
fi
}

function copyLibs() {
local DEST=$1 && shift
local DEST_PATH=$(cygpath -u "${DEST}")
for lib in ${LIBS[@]}; do
echo "copy ${lib} to destination"
cp "$(which "${lib}")" "${DEST_PATH}"
done
}

getLibsForBinary "${TARGET}/canvas.node"
copyLibs "${TARGET}"
38 changes: 20 additions & 18 deletions prebuild/Windows/preinstall.sh
Original file line number Diff line number Diff line change
@@ -1,34 +1,36 @@
#!/usr/bin/env sh
# expects node, VS, and MSYS environments to be set up already. does everything else.

deps="cairo-2 png16-16 jpeg-8 pango-1.0-0 pangocairo-1.0-0 gobject-2.0-0 glib-2.0-0 turbojpeg gif-7 freetype-6 rsvg-2-2 gsf-1-114";
deps="cairo-2 png16-16 jpeg-8 pango-1.0-0 pangocairo-1.0-0 gobject-2.0-0 glib-2.0-0 turbojpeg gif-7 freetype-6 rsvg-2-2";

# install cairo and tools to create .lib

pacman --noconfirm -S \
wget \
unzip \
ucrt64/mingw-w64-ucrt-x86_64-binutils \
ucrt64/mingw-w64-ucrt-x86_64-tools \
ucrt64/mingw-w64-ucrt-x86_64-libjpeg-turbo \
ucrt64/mingw-w64-ucrt-x86_64-pango \
ucrt64/mingw-w64-ucrt-x86_64-cairo \
ucrt64/mingw-w64-ucrt-x86_64-giflib \
ucrt64/mingw-w64-ucrt-x86_64-freetype \
ucrt64/mingw-w64-ucrt-x86_64-fontconfig \
ucrt64/mingw-w64-ucrt-x86_64-librsvg \
ucrt64/mingw-w64-ucrt-x86_64-libxml2 \
ucrt64/mingw-w64-ucrt-x86_64-libgsf
prefix=${MSYSTEM,,}
arch=${MSYSTEM_CARCH}

pacman --noconfirm --needed -S \
${prefix}/mingw-w64-ucrt-${arch}-binutils \
${prefix}/mingw-w64-ucrt-${arch}-tools \
${prefix}/mingw-w64-ucrt-${arch}-libjpeg-turbo \
${prefix}/mingw-w64-ucrt-${arch}-pango \
${prefix}/mingw-w64-ucrt-${arch}-cairo \
${prefix}/mingw-w64-ucrt-${arch}-giflib \
${prefix}/mingw-w64-ucrt-${arch}-harfbuzz \
${prefix}/mingw-w64-ucrt-${arch}-freetype \
${prefix}/mingw-w64-ucrt-${arch}-fontconfig \
${prefix}/mingw-w64-ucrt-${arch}-librsvg \
${prefix}/mingw-w64-ucrt-${arch}-libxml2

# create .lib files for vc++

echo "generating lib files for the MSYS2 UCRT64 dlls"
echo "generating lib files for the MSYS2 dlls"
for lib in $deps; do
gendef /ucrt64/bin/lib$lib.dll > /dev/null 2>&1 || {
gendef /${prefix}/bin/lib$lib.dll > /dev/null 2>&1 || {
echo "could not find lib$lib.dll, have to skip ";
continue;
}

dlltool -d lib$lib.def -l /ucrt64/lib/lib$lib.lib > /dev/null 2>&1 || {
dlltool -d lib$lib.def -l /${prefix}/lib/lib$lib.lib > /dev/null 2>&1 || {
echo "could not create dll for lib$lib.dll";
continue;
}
Expand Down
Loading