Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APK is not reproducible #898

Closed
emanuelb opened this issue Mar 14, 2022 · 5 comments
Closed

APK is not reproducible #898

emanuelb opened this issue Mar 14, 2022 · 5 comments
Labels
Bug Something isn't working

Comments

@emanuelb
Copy link

Related issues are #463 (diffs from compare f-droid APK to Github/GooglePlay APK) and #416, tried to build version 0.6.0 with Containerfile

FROM debian:sid-slim

RUN set -ex; \
    mkdir -p /usr/share/man/man1/; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
        npm \
        git \
        openjdk-11-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser;

USER appuser

ENV NODE_ENV="development" \
    ANDROID_SDK_ROOT="/home/appuser/app/sdk/" \
    ANDROID_HOME="/home/appuser/app/sdk/"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/; \
    git clone https://github.com/ZeusLN/zeus;

WORKDIR /home/appuser/app/zeus/

RUN set -ex; \
    git checkout v0.6.0; \
    cd /home/appuser/app/zeus/android; \
    keytool -genkey -alias zeus_alias -keystore app/zeus.pfx -storetype PKCS12 -keyalg RSA -keysize 4096 -storepass zeus_alias -keypass zeus_alias -validity 10000 -dname CN=IL; \
    printf "\nMYAPP_RELEASE_STORE_FILE=zeus.pfx\nMYAPP_RELEASE_STORE_PASSWORD=zeus_alias\nMYAPP_RELEASE_KEY_PASSWORD=zeus_alias\nMYAPP_RELEASE_KEY_ALIAS=zeus_alias\n" >> gradle.properties;
    
RUN set -ex; \
    npm install; \
    cd /home/appuser/app/zeus/android; \
    ./gradlew assembleRelease

compared to APK at: https://github.com/ZeusLN/zeus/releases/download/v0.6.0/zeus-v0.6.0.apk
sha256sum:4e556a12a6730a75574bdecf37c1537d6578ea55271371b59ca501e922a465dc

Generated APK in: /home/appuser/app/zeus/android/app/build/outputs/apk/release/app-universal-release.apk

Result is:

Files ./zeus-v0.6.0/assets/index.android.bundle and ./zeus6-apk-rebuild/assets/index.android.bundle differ
Files ./zeus-v0.6.0/res/23.xml and ./zeus6-apk-rebuild/res/23.xml differ
Files ./zeus-v0.6.0/res/AP.xml and ./zeus6-apk-rebuild/res/AP.xml differ
Files ./zeus-v0.6.0/res/color/material_on_background_disabled.xml and ./zeus6-apk-rebuild/res/color/material_on_background_disabled.xml differ
Files ./zeus-v0.6.0/res/color/material_on_background_emphasis_high_type.xml and ./zeus6-apk-rebuild/res/color/material_on_background_emphasis_high_type.xml differ
Files ./zeus-v0.6.0/res/color/material_on_background_emphasis_medium.xml and ./zeus6-apk-rebuild/res/color/material_on_background_emphasis_medium.xml differ
Files ./zeus-v0.6.0/res/color/material_on_primary_disabled.xml and ./zeus6-apk-rebuild/res/color/material_on_primary_disabled.xml differ
Files ./zeus-v0.6.0/res/color/material_on_primary_emphasis_high_type.xml and ./zeus6-apk-rebuild/res/color/material_on_primary_emphasis_high_type.xml differ
Files ./zeus-v0.6.0/res/color/material_on_primary_emphasis_medium.xml and ./zeus6-apk-rebuild/res/color/material_on_primary_emphasis_medium.xml differ
Files ./zeus-v0.6.0/res/gT.xml and ./zeus6-apk-rebuild/res/gT.xml differ
Files ./zeus-v0.6.0/res/Hh.xml and ./zeus6-apk-rebuild/res/Hh.xml differ
Files ./zeus-v0.6.0/res/J3.png and ./zeus6-apk-rebuild/res/J3.png differ
Files ./zeus-v0.6.0/res/Jc.xml and ./zeus6-apk-rebuild/res/Jc.xml differ
Files ./zeus-v0.6.0/res/jm.xml and ./zeus6-apk-rebuild/res/jm.xml differ
Files ./zeus-v0.6.0/res/jR.xml and ./zeus6-apk-rebuild/res/jR.xml differ
Files ./zeus-v0.6.0/res/lJ.xml and ./zeus6-apk-rebuild/res/lJ.xml differ
Files ./zeus-v0.6.0/res/qQ.xml and ./zeus6-apk-rebuild/res/qQ.xml differ
Files ./zeus-v0.6.0/res/r-.xml and ./zeus6-apk-rebuild/res/r-.xml differ
Files ./zeus-v0.6.0/res/TK.xml and ./zeus6-apk-rebuild/res/TK.xml differ
Files ./zeus-v0.6.0/res/v8.xml and ./zeus6-apk-rebuild/res/v8.xml differ
Files ./zeus-v0.6.0/res/vH.xml and ./zeus6-apk-rebuild/res/vH.xml differ
Files ./zeus-v0.6.0/res/vp.xml and ./zeus6-apk-rebuild/res/vp.xml differ
Files ./zeus-v0.6.0/res/VW.xml and ./zeus6-apk-rebuild/res/VW.xml differ
Files ./zeus-v0.6.0/res/wA.xml and ./zeus6-apk-rebuild/res/wA.xml differ
Files ./zeus-v0.6.0/res/wd.xml and ./zeus6-apk-rebuild/res/wd.xml differ
Files ./zeus-v0.6.0/res/x1.xml and ./zeus6-apk-rebuild/res/x1.xml differ
Files ./zeus-v0.6.0/res/z3.xml and ./zeus6-apk-rebuild/res/z3.xml differ
@emanuelb
Copy link
Author

emanuelb commented Aug 4, 2022

trying to reproduce version 0.6.5
compile with: podman build --pull --rm -t zeus_build_apk -f Containerfile

Containerfile:

FROM debian:sid-slim

RUN set -ex; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
        npm \
        git \
        openjdk-11-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser;

USER appuser

ENV NODE_ENV="development" \
    ANDROID_SDK_ROOT="/home/appuser/app/sdk/" \
    ANDROID_HOME="/home/appuser/app/sdk/"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/; \
    git clone https://github.com/ZeusLN/zeus;

WORKDIR /home/appuser/app/zeus/

RUN set -ex; \
    git checkout v0.6.5; \
    cd /home/appuser/app/zeus/android; \
    keytool -genkey -alias zeus_alias -keystore app/zeus.pfx -storetype PKCS12 -keyalg RSA -keysize 4096 -storepass zeus_alias -keypass zeus_alias -validity 10000 -dname CN=IL; \
    printf "\nMYAPP_RELEASE_STORE_FILE=zeus.pfx\nMYAPP_RELEASE_STORE_PASSWORD=zeus_alias\nMYAPP_RELEASE_KEY_PASSWORD=zeus_alias\nMYAPP_RELEASE_KEY_ALIAS=zeus_alias\n" >> gradle.properties;
    
RUN set -ex; \
    npm install --force;  \
    cd /home/appuser/app/zeus/android; \
    ./gradlew assembleRelease

--force parameter was used for npm install cause without it the error happens:

npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR! 
npm ERR! While resolving: @react-native-async-storage/[email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/react-native
npm ERR!   react-native@"0.68.2" from the root project
npm ERR!   peer react-native@">=0.57.0" from @react-native-clipboard/[email protected]
npm ERR!   node_modules/@react-native-clipboard/clipboard
npm ERR!     @react-native-clipboard/clipboard@"1.9.0" from the root project
npm ERR!   22 more (@react-native-community/cli, ...)
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer react-native@"^0.0.0-0 || 0.60 - 0.67 || 1000.0.0" from @react-native-async-storage/[email protected]
npm ERR! node_modules/@react-native-async-storage/async-storage
npm ERR!   @react-native-async-storage/async-storage@"1.15.17" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: [email protected]
npm ERR! node_modules/react-native
npm ERR!   peer react-native@"^0.0.0-0 || 0.60 - 0.67 || 1000.0.0" from @react-native-async-storage/[email protected]
npm ERR!   node_modules/@react-native-async-storage/async-storage
npm ERR!     @react-native-async-storage/async-storage@"1.15.17" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! See /home/appuser/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/appuser/.npm/_logs/2022-08-04T00_10_33_420Z-debug-0.log
Error: error building at STEP "RUN set -ex;     npm install;     cd /home/appuser/app/zeus/android;     ./gradlew assembleRelease": error while running runtime: exit status 1

comparing generated APK to:
https://github.com/ZeusLN/zeus/releases/download/v0.6.5/zeus-v0.6.5.apk

Files ./FromBuild/assets/index.android.bundle and ./FromGithub/assets/index.android.bundle differ
Files ./FromBuild/res/23.xml and ./FromGithub/res/23.xml differ
Files ./FromBuild/res/AP.xml and ./FromGithub/res/AP.xml differ
Files ./FromBuild/res/Hh.xml and ./FromGithub/res/Hh.xml differ
Files ./FromBuild/res/J3.png and ./FromGithub/res/J3.png differ
Files ./FromBuild/res/Jc.xml and ./FromGithub/res/Jc.xml differ
Files ./FromBuild/res/TK.xml and ./FromGithub/res/TK.xml differ
Files ./FromBuild/res/VW.xml and ./FromGithub/res/VW.xml differ
Files ./FromBuild/res/color/material_on_background_disabled.xml and ./FromGithub/res/color/material_on_background_disabled.xml differ
Files ./FromBuild/res/color/material_on_background_emphasis_high_type.xml and ./FromGithub/res/color/material_on_background_emphasis_high_type.xml differ
Files ./FromBuild/res/color/material_on_background_emphasis_medium.xml and ./FromGithub/res/color/material_on_background_emphasis_medium.xml differ
Files ./FromBuild/res/color/material_on_primary_disabled.xml and ./FromGithub/res/color/material_on_primary_disabled.xml differ
Files ./FromBuild/res/color/material_on_primary_emphasis_high_type.xml and ./FromGithub/res/color/material_on_primary_emphasis_high_type.xml differ
Files ./FromBuild/res/color/material_on_primary_emphasis_medium.xml and ./FromGithub/res/color/material_on_primary_emphasis_medium.xml differ
Files ./FromBuild/res/gT.xml and ./FromGithub/res/gT.xml differ
Files ./FromBuild/res/jR.xml and ./FromGithub/res/jR.xml differ
Files ./FromBuild/res/jm.xml and ./FromGithub/res/jm.xml differ
Files ./FromBuild/res/lJ.xml and ./FromGithub/res/lJ.xml differ
Files ./FromBuild/res/qQ.xml and ./FromGithub/res/qQ.xml differ
Files ./FromBuild/res/r-.xml and ./FromGithub/res/r-.xml differ
Files ./FromBuild/res/v8.xml and ./FromGithub/res/v8.xml differ
Files ./FromBuild/res/vH.xml and ./FromGithub/res/vH.xml differ
Files ./FromBuild/res/vp.xml and ./FromGithub/res/vp.xml differ
Files ./FromBuild/res/wA.xml and ./FromGithub/res/wA.xml differ
Files ./FromBuild/res/wd.xml and ./FromGithub/res/wd.xml differ
Files ./FromBuild/res/x1.xml and ./FromGithub/res/x1.xml differ
Files ./FromBuild/res/z3.xml and ./FromGithub/res/z3.xml differ

@emanuelb
Copy link
Author

looks like a solution is WIP at PR: #1055

@kaloudis
Copy link
Contributor

This is now solved with the addition of reproducible builds https://github.com/ZeusLN/zeus#reproducible-builds

@kaloudis kaloudis closed this as completed Oct 4, 2022
@emanuelb
Copy link
Author

emanuelb commented Feb 7, 2023

While Issue is solved as APK can be reproduced by written Containerfile below that is based on build.sh script https://github.com/ZeusLN/zeus/blob/a1828c8c3fad9d11364723fedab59dc62ccd1d30/build.sh

several improvements are:

  1. make the build not depend on specific build path /olympus/zeus and node version (which currently it's old and unsupported 14.19.0) details in issue: Filepath & node version are leaked into apk in index.android.bundle file #463
  2. use newer tools and smaller base image (such as latest ubuntu LTS, and supported NDK version instead of old version 21)

build.sh script use base image reactnativecommunity/react-native-android:5.4 which is 1 year old:
https://hub.docker.com/layers/reactnativecommunity/react-native-android/5.4/images/sha256-e3120cdd97f76061d3c0bd7d5615ed01846e9a236f791f40aa24d910d935558f?context=explore

and based on ubuntu:20.04 with additions from: https://github.com/react-native-community/docker-android/blob/1dd65ea0aacfa91f87ad9df531ca85d244838052/Dockerfile#L1

that install a lot of unnecessary dependencies that wont be installed in the below minimal Containerfile

build with: podman build --rm --ulimit nofile=8192:8192 -t zeus_rb_build_ubuntu_base -f ContainerFileUbuntuRB702

FROM docker.io/ubuntu:20.04

RUN set -ex; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
      curl \
      unzip \
      openjdk-11-jdk-headless; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser; \
    mkdir -p /olympus/ /opt/node/ /opt/yarn/ /opt/android/ndk/ /opt/android/licenses; \
    chown appuser:appuser /olympus/; \
    chown -R appuser:appuser /opt/;

USER appuser

ENV NODE_ENV="development" \
    ANDROID_SDK_ROOT="/opt/android" \
    ANDROID_HOME="/opt/android" \
    PATH="/opt/android/ndk/21.4.7075529/:/opt/node/bin/:/opt/yarn/node_modules/.bin/:$PATH"

USER appuser
WORKDIR /olympus/

RUN set -ex; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/opt/android/licenses/android-sdk-license"; \
    curl -o v0.7.2.zip -L https://github.com/ZeusLN/zeus/archive/refs/tags/v0.7.2.zip; \
    unzip v0.7.2.zip -d /olympus/; \ 
    rm v0.7.2.zip; \
    mv /olympus/zeus-0.7.2 /olympus/zeus; \
    cd /opt/android/ndk/; \
    curl -o ndk21.zip -L https://dl.google.com/android/repository/android-ndk-r21e-linux-x86_64.zip; \
    echo "ad7ce5467e18d40050dc51b8e7affc3e635c85bd8c59be62de32352328ed467e  ndk21.zip" | sha256sum -c ; \
    unzip ndk21.zip; \
    rm ndk21.zip; \
    mv android-ndk-r21e /opt/android/ndk/21.4.7075529/; \
    cd /opt/node/; \
    curl -o node-v14.19.0-linux-x64.tar.gz -L https://nodejs.org/download/release/v14.19.0/node-v14.19.0-linux-x64.tar.gz; \
    echo "223ca31e3440b79a3fe6828161b1843743eaa7610a88c0e1ac1d8e1d815b44cd  node-v14.19.0-linux-x64.tar.gz" | sha256sum -c; \
    tar -xf node-v14.19.0-linux-x64.tar.gz --strip-components=1; \
    rm node-v14.19.0-linux-x64.tar.gz; \
    cd /opt/yarn/; \
    npm install yarn; \
    cd /olympus/zeus; \
    yarnpkg install --frozen-lockfile; \
    cd /olympus/zeus/node_modules/@lightninglabs/lnc-rn; \
    bash fetch-libraries.sh; \
    cd /olympus/zeus/android; \
    ./gradlew assembleRelease;

Compared with diff --recursive --brief ./FromLocal ./FromGithub to apk from github: https://github.com/ZeusLN/zeus/releases/download/v0.7.2/zeus-v0.7.2.apk

Only in ./FromGithub/META-INF: MANIFEST.MF
Only in ./FromGithub/META-INF: ZEUS-KEY.RSA
Only in ./FromGithub/META-INF: ZEUS-KEY.SF

@kaloudis
Copy link
Contributor

kaloudis commented Feb 8, 2023

Glad to see you were able to reproduce! We're updating some things here: #1310

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants