nix: add flake and derivation #275
Open
+184
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Referenced issue: waku-org/nwaku#3232 Signed-off-by: markoburcul <[email protected]>
|
What needs to be checked is what to do with the rest of these architectures and systems from the list in Cross.toml file. From what I know nix supports following list. |
Benchmark for ef87b27Click to view benchmark
|
Benchmark for ef87b27Click to view benchmark
|
Benchmark for ef87b27Click to view benchmark
|
|
I don't think we should worry about cross compiling. I don't see any scenario in which we have to cross-compile |
For android I currently cross compile zerokit and status-go to the following architectures
This is being done thru the targets |
|
There's another possibility, which is to cross compile zerokit without using cross-rs, like it's described in https://rust-lang.github.io/rustup/cross-compilation.html . In theory we should add a .cargo/config.toml file describing the toolchains to use depending on the arch, for example: And assuming you have the android NDK toolchain in the $PATH, it should work |
|
Oh, right, I'm dumb. For android we would need to cross compile. But with Nix there's two ways to do this:
|
I've added this in my newest commit for cross compiling but only for |
Benchmark for 2fdda1bClick to view benchmark
|
Benchmark for 2fdda1bClick to view benchmark
|
Benchmark for 2fdda1bClick to view benchmark
|
Referenced issue: waku-org/nwaku#3232 Signed-off-by: markoburcul <[email protected]>
markoburcul
force-pushed
the
add-nix-flake-and-derivation
branch
from
2dd815f to
0474219
Compare
Benchmark for 3ee9c5fClick to view benchmark
|
Benchmark for 3ee9c5fClick to view benchmark
|
Benchmark for 3ee9c5fClick to view benchmark
|
|
The problem I am encountering is that we need to get the shared object file as the result but unfortunately with this initial version of the flake that is not possible.The only thing I get is: [nix-shell:~/work/zerokit]$ tree result
result
├── bin
│ └── rln-cli
└── lib
└── librln.a
3 directories, 2 filesWithin the scripts from nwaku that build this with cross it is using some combination of rustc and cargo flags which is not possible with just cargo or just rustc. Issuing this command within the I expected that when building with |
|
After discussion with @richard-ramos I've adjusted the build commands and managed to build shared object library with some minor tweaks to the derivation: buildPhase = ''
pushd rln
cargo rustc --crate-type=cdylib --release --lib
popd
'';
installPhase = ''
mkdir -p $out/build/
cp -r ./target/release/librln.so $out/build/
'';The next step is to test cross compilation. |
|
I've tried on my laptop to cross compile using: pkgs.pkgsCross.${target-platform}.rustPlatform.buildRustPackagebut it blocked after it finished build phase for the platform binaries(?) in 30min which is tragic. I'll try it on one of our CI hosts which are more beefy, but if this doesn't work I'll have to think of providing all the packages depending on the target os/arch within the derivation. |
|
What do you mean it blocked? It got stuck? Was it doing anything(strace?) or just hanged? |
I didn't look with strace, it just hanged with no output and then I killed it |
|
Well, trying on a bigger CI host is a good idea, especially since this is something that probably will rarely change and we can just cache in our Nix binary cache. But it would be interesting to know if this was a resource problem or some bug if it happens again. |
Signed-off-by: markoburcul <[email protected]>
markoburcul
force-pushed
the
add-nix-flake-and-derivation
branch
from
8d4f36d to
d6675af
Compare
I went with path to create a minimal Dockerfile and compile it in container to understand all of the components necessary for compiling the zerokit shared object library: FROM debian:buster
# Install Rust
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
ca-certificates \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN curl https://sh.rustup.rs -sSf | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
# Install required dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates-java \
openjdk-11-jdk \
wget \
unzip \
build-essential \
clang \
make \
gcc \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Set environment variables for Android NDK
ENV ANDROID_NDK_HOME=/android-ndk
ENV PATH="$ANDROID_NDK_HOME:$PATH"
# Download and install the Android NDK
ARG ANDROID_NDK_VERSION=r27c
RUN wget https://dl.google.com/android/repository/android-ndk-${ANDROID_NDK_VERSION}-linux.zip \
&& unzip android-ndk-${ANDROID_NDK_VERSION}-linux.zip -d / \
&& mv /android-ndk-${ANDROID_NDK_VERSION} $ANDROID_NDK_HOME \
&& rm android-ndk-${ANDROID_NDK_VERSION}-linux.zip
# Install Rust targets for Android
RUN rustup target add armv7-linux-androideabi
ENV CC=/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi25-clang
ENV CXX=/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi25-clang++
ENV CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER=/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi25-clang
# Set working directory
WORKDIR /workspace
# Default command
CMD ["/bin/bash"]run it inside zerokit repo: docker run --rm -it -v ./:/workspace cross_rust:latest bashcompile inside container: cargo rustc --crate-type=cdylib --release --lib --target=armv7-linux-androideabiI think I need to figure out how to point to the right linker from android ndk using the env variable |
markoburcul
force-pushed
the
add-nix-flake-and-derivation
branch
from
3d4aeae to
1ad8b55
Compare
Benchmark for 5c008ebClick to view benchmark
|
Benchmark for 114376bClick to view benchmark
|
|
I've managed to cross compile for Another thing I tried is to cross compile the code, in my docker container from above, without using rustup to install targets and the error is basically related to std not being there which is expected: What I don't like is that I still haven't found a way to mitigate this issue and usage of rustup. |
markoburcul
force-pushed
the
add-nix-flake-and-derivation
branch
from
1ad8b55 to
b17d9a2
Compare
Signed-off-by: markoburcul <[email protected]>
markoburcul
force-pushed
the
add-nix-flake-and-derivation
branch
from
b17d9a2 to
4479810
Compare
|
I've managed to eliminate rustup from the equation, as an example this is the Dockerfile: # Use a base image with a minimal installation of Ubuntu
FROM ubuntu:20.04
# Set the working directory
WORKDIR /tmp
# Update and install required dependencies
RUN apt-get update && apt-get install -y \
curl \
wget \
unzip \
xz-utils \
build-essential \
clang \
make \
gcc \
file
# Download necessary files
RUN wget https://dl.google.com/android/repository/android-ndk-r28-linux.zip && \
wget https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init && \
wget https://static.rust-lang.org/dist/2025-01-30/rust-std-1.84.1-x86_64-linux-android.tar.gz && \
wget https://static.rust-lang.org/dist/rust-1.84.1-x86_64-unknown-linux-gnu.tar.xz && \
wget https://static.rust-lang.org/dist/2025-01-30/rust-std-1.84.1-aarch64-linux-android.tar.gz
# Extract the Android NDK
RUN unzip android-ndk-r28-linux.zip -d /android-ndk/ && \
mv /android-ndk/android-ndk-r28/* /android-ndk/ && \
rm -rf /android-ndk/android-ndk-r28
# Extract and install the Rust toolchain
RUN tar -xvzf rust-std-1.84.1-x86_64-linux-android.tar.gz && \
tar -xvzf rust-std-1.84.1-aarch64-linux-android.tar.gz && \
tar -xvf rust-1.84.1-x86_64-unknown-linux-gnu.tar.xz && \
./rust-1.84.1-x86_64-unknown-linux-gnu/install.sh --prefix=$HOME/rust-install/rust-1.84.1-x86_64-unknown-linux-gnu && \
./rust-std-1.84.1-x86_64-linux-android/install.sh --prefix=$HOME/rust-install/rust-1.84.1-x86_64-unknown-linux-gnu && \
./rust-std-1.84.1-aarch64-linux-android/install.sh --prefix=$HOME/rust-install/rust-1.84.1-x86_64-unknown-linux-gnu
# Set up Rust toolchain
RUN chmod +x rustup-init && \
./rustup-init --default-toolchain none -y && \
. "$HOME/.cargo/env" && \
rustup toolchain link rust-toolchain-1.84.1 $HOME/rust-install/rust-1.84.1-x86_64-unknown-linux-gnu && \
rustup default rust-toolchain-1.84.1
# Export environment variables for Android NDK
RUN export CC=/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android25-clang && \
export CXX=/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android25-clang++ && \
export CARGO_TARGET_X86_64_LINUX_ANDROID_LINKER=/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android25-clang
# Set up working directory for your project
WORKDIR /workspace
CMD ["bash"]Inside of the container you can easily compile the RLN shared object library. The only thing that needs to be changed when compiling are CC, CXX and CARGO_TARGET__LINKER variable and their values. The issue is that in Nix derivation you can't run bash scripts since env is not available so removing rustup from the equation is not a valid approach. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Referenced issue: waku-org/nwaku#3232