Please add aarch64 (ARM 64-bit) builds

[probonopd]

Please add aarch64 (ARM 64-bit) builds.

https://github.com/pypa/manylinux seems to have a Docker container for it.

[niess]

Indeed. I didn't do it so far because I don't know how to do the cross build.

[probonopd]

Travis CI as aarch64 workers but I don't know about GitHub Flows.

[niess]

OK, I could do it on travis then. I was looking at the manylinux repo how they generate these images but I can't find any reference to aarch64. It looks like they only build the x86_64 and i686 images there?

[probonopd]

https://github.com/pypa/manylinux, search for aarch64.

[niess]

Got it finally. It's in a specific branch, the manylinux2014. They are using Travis as well as you suggested. The only issue left is how to merge the results from GitHub workflow & Travis in a single release, e.g. having x86_64 (GitHub workflow) and aarch64 (Travis) published under a single python3.8 tag/release.

But anyway, in the worst case I could publish them separately.

[netphantom]

Hi, I reopen this issue because I am trying to build an appimage for armv7.
I want to point out that I am able to correctly build my program for amd64 using python-appimage without any additional tags, but maybe I am missing something in the procedure to build the armv7.

I have tried to download and use the manylinux_2014 appimages that are in the release page here, but actually I am having issues when passing them to python-appimage. Could somebody please help me to figure out the correct procedure to follow to build the app for armv7?

Thanks in advance

[niess]

Hello @netphantom,

Unfortunately, armv7 is not yet supported. You can check the documentation for available combinations of Python versions, Manylinux tags and architectures.

Manylinux Docker images for armv7 do exist. However, the problem is that I didn't succeed so far in cross-packaging those as an AppImage using GitHub CI. The last time that I tried, the resulting AppImages crashed when running them through an armv7 emulator (qemu). Though, I do not know if this was an AppImage issue or an emulator one.

If someone has experience with this topic, I am very open to a PR. Note that one would first need to produce an amrv7 AppImage of patchelf. My unsuccessful attempt at this can be found under the dev branch of the latter project.

[ForBai]

any updates on this?

[mgmartian]

Hi @niess. My employer uses the app images for some internal tools; as a way of running newer Pythons on older OSes with little hassle. We might be interested in sponsoring (paying) for you to do the work to add support for aarm64.

I didn't see an email address on your profile, so I'm reaching out here. Could you contact me, please?

Thanks.
(email removed)
^^ This is a throw-away account, to avoid getting SPAM at my corporate email address.

[niess]

Hello @mgmartian,

thanks for proposing :)

I am registered on GitHub with my company (CNRS) email. This is why, I am reluctant to expose it. But, my Gmail is visible on PyPI (in the "creator" metadata).

The main issue for me is time. Currently, there does not seem to be a straightforward workflow using GitHub CI. I tried, but failed, through emulation (with qemu). I could use another provider (e.g. numpy seems to rely on Cirrus CI). But, then I'll have to maintain two workflows, which I would rather not (since time is an issue, and I doubt that I would be "sponsored" for maintaining these workflows :P).

That said, I am willing to make a new attempt (e.g. over a week-end, out of my working hours). Things might have evolved since my last attempt. But, one more complication is that I do not have a native ARM for testing the produced AppImages :(

[mgmartian]

Hi @niess,

I really appreciate all of your work on x86 -- it has saved us a lot of time / trouble in our work. 🤩. I certainly understand not wanting to spend the time / money (!) on this; particularly if it's not something you personally need.

Probably I'm going to have to do this on my own, internally to our company. ☹️ If I have any insights, I will share them here.

Here are a few ideas, if you / someone else is going to tackle this:

Idea 1:
GitHub has ARM Linux runners now but I think these are considered "larger runners" which may not be available to a "free" account if that's what you've got.

Idea 2:
Another option might be to spin up an AWS ARM VM ("gravitron") and then connect it to GitHub as a "self-hosted" GitHub runner. You could also use this box to test. Leaving this box turned on 24 x 7 x 365 is probably not going to be free though. Might be free to "turn it on", run some tests for a few hours, then "turn it off".

[niess]

@mgmartian,

I indeed have a "free" account, and therefore no access to GitHub ARM runners. But anyway, I might have a more simple solution, which is basically "cross-building".

Actually, it is possible to extract the Manylinux Python runtimes from an ARM Docker image, but using a x86_64 host. The idea is not to run the Manylinux Docker image, but to simply dump its content to a folder, and then to relocate (extract) the Python runtime. In addition, one has to edit the RPATH of ELF files to match the relocation. And, it looks like patchelf actually can edit ELF files for another architecture.

I tested this idea yesterday (the proof-of-concept is over there), and it seems to work on my side (emulating ARM with qemu). That is, I could run (through qemu) an aarch64 Python runtime extracted from my x86_64 host. To complete this proof-of-concept, it would be great if you could check that the extracted runtime (here) properly works on a native ARM. If you do have access to this?

[williamkborn]

I just pulled down your extracted runtime and tested on native apple silicon with fedora 40.

$ lsb_release -a
LSB Version:    n/a
Distributor ID: Fedora
Description:    Fedora Linux 40 (KDE Plasma)
Release:        40
Codename:       n/a
$ uname -a
Linux fedora 6.8.9-300.fc40.aarch64 #1 SMP PREEMPT_DYNAMIC Thu May  2 19:17:51 UTC 2024 aarch64 GNU/Linux
$ ./bin/python3
python3     python3.11  
$ ./bin/python3.11
Python 3.11.9 (main, Jun  6 2024, 22:42:58) [GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("hello")
hello
>>> exit()
$ md5sum ../python3.11-2014_aarch64.tar.gz 
553868bcea82e86eebd4b82372f30d94  ../python3.11-2014_aarch64.tar.gz

Seems to work great.

[mgmartian]

Awesome @niess! What a pleasant surprise! I was able to download this tarball and run it on an ARM Neoverse N1 system we have running Rocky Linux 9.3. I built a venv, installed a few packages and poked around. Seemed to work great!

[niess]

Thanks for testing @williamkborn @mgmartian .

I would call this a success then :)

Now, I'll have to see how to properly integrate this strategy into python-appimage, without breaking it, which might take some more time.

Meanwhile, you might use the extract-manylinux example for manually producing tarballs of the Python runtime (I added a few instructions that for). Note however that, packaging those tarballs as AppImages would not be 100% functional (e.g. concerning venv). But, the extracted tarballs should work.

[awesomebytes]

Hello,
tested the build linked before (https://github.com/niess/extract-manylinux/releases/download/rolling/python3.11-2014_aarch64.tar.gz) on another aarch64 system (a nvidia orin) and it worked too.