Building Docker images on Apple M3

[camsellem]

Hi

I've been trying to build the demo for linux/amd64 architecture on my Macbook Air M3. Although I'm struggling with a couple of services that fail with errors.

I'm using the command docker compose build servicename with the --platform flag set on linux/amd64 via the environment variable DOCKER_DEFAULT_PLATFORM

The two services that fail are Frauddetectionservice and Shippingservice

The error for Frauddetectionservice is the following

 => ERROR [frauddetectionservice builder 5/5] RUN gradle shadowJar                                                                                                                                        7.8s
------
 > [frauddetectionservice builder 5/5] RUN gradle shadowJar:
1.297 Starting a Gradle Daemon (subsequent builds will be faster)
7.696 The message received from the daemon indicates that the daemon has disappeared.
7.696 Build request sent: Build{id=f63da213-91b6-4847-b8b6-0fd0f71cd353, currentDir=/usr/src/app}
7.696 Attempting to read last messages from the daemon log...
7.696 Daemon pid: 43
7.696   log file: /home/gradle/.gradle/daemon/8.8/daemon-43.out.log
7.696 ----- Last  20 lines from daemon log file - daemon-43.out.log -----
7.697 2024-06-24T04:32:39.965+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] Command execution: started DaemonCommandExecution[command = Build{id=f63da213-91b6-4847-b8b6-0fd0f71cd353, currentDir=/usr/src/app}, connection = DefaultDaemonConnection: socket connection from /127.0.0.1:45847 to /127.0.0.1:45170] after 0.0 minutes of idle
7.697 2024-06-24T04:32:39.965+0000 [INFO] [org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Marking the daemon as busy, address: [63fdfa25-c9d8-44f5-99a0-bec05fa04702 port:45847, addresses:[localhost/127.0.0.1]]
7.697 2024-06-24T04:32:39.965+0000 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Marking busy by address: [63fdfa25-c9d8-44f5-99a0-bec05fa04702 port:45847, addresses:[localhost/127.0.0.1]]
7.697 2024-06-24T04:32:39.966+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on daemon addresses registry.
7.697 2024-06-24T04:32:39.967+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
7.698 2024-06-24T04:32:39.968+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
7.698 2024-06-24T04:32:39.968+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] resetting idle timer
7.698 2024-06-24T04:32:39.968+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running. Sleeping until state changes.
7.698 2024-06-24T04:32:39.969+0000 [INFO] [org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is about to start building Build{id=f63da213-91b6-4847-b8b6-0fd0f71cd353, currentDir=/usr/src/app}. Dispatching build started information...
7.698 2024-06-24T04:32:39.969+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 23: dispatching org.gradle.launcher.daemon.protocol.BuildStarted@298f500
7.699 2024-06-24T04:32:39.970+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring env variables: [PATH, LANGUAGE, LC_ALL, GRADLE_VERSION, JAVA_HOME, PWD, JAVA_VERSION, LANG, HOME, GRADLE_HOME]
7.701 2024-06-24T04:32:39.971+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 21: received class org.gradle.launcher.daemon.protocol.CloseInput
7.701 2024-06-24T04:32:39.972+0000 [DEBUG] [org.gradle.launcher.daemon.server.DefaultDaemonConnection] thread 21: Received IO message from client: org.gradle.launcher.daemon.protocol.CloseInput@1e3c83ed
7.701 2024-06-24T04:32:39.975+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying all logs to the client via the connection.
7.701 2024-06-24T04:32:39.975+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogToClient] The client will now receive all logging from the daemon (pid: 43). The daemon log file: /home/gradle/.gradle/daemon/8.8/daemon-43.out.log
7.701 2024-06-24T04:32:39.977+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogAndCheckHealth] Starting build in new daemon [memory: 512 MiB]
7.701 2024-06-24T04:32:39.978+0000 [DEBUG] [org.gradle.launcher.daemon.server.clientinput.ClientInputForwarder] Closing this process' stdin at end of input.
7.701 2024-06-24T04:32:39.978+0000 [DEBUG] [org.gradle.launcher.daemon.server.clientinput.ClientInputForwarder] This process will no longer process any forwarded input.
7.701 2024-06-24T04:32:39.978+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has started executing the build.
7.701 2024-06-24T04:32:39.978+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] Executing build with daemon context: DefaultDaemonContext[uid=a18921b8-e9c8-4caa-ace8-06a1029c52f2,javaHome=/opt/java/openjdk,javaVersion=17,daemonRegistryDir=/home/gradle/.gradle/daemon,pid=43,idleTimeout=10800000,priority=NORMAL,applyInstrumentationAgent=true,nativeServicesMode=ENABLED,daemonOpts=--add-opens=java.base/java.util=ALL-UNNAMED,--add-opens=java.base/java.lang=ALL-UNNAMED,--add-opens=java.base/java.lang.invoke=ALL-UNNAMED,--add-opens=java.prefs/java.util.prefs=ALL-UNNAMED,--add-opens=java.base/java.nio.charset=ALL-UNNAMED,--add-opens=java.base/java.net=ALL-UNNAMED,--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED,-XX:MaxMetaspaceSize=384m,-XX:+HeapDumpOnOutOfMemoryError,-Xms256m,-Xmx512m,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]
7.701 ----- End of the daemon log -----
7.701
7.701 JVM crash log found: file:///usr/src/app/hs_err_pid43.log
7.708
7.708 FAILURE: Build failed with an exception.
7.708
7.708 * What went wrong:
7.709 Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
7.710
7.710 * Try:
7.710 > Run with --stacktrace option to get the stack trace.
7.711 > Run with --info or --debug option to get more log output.
7.711 > Run with --scan to get full insights.
7.711 > Get more help at https://help.gradle.org.
------
failed to solve: process "/bin/sh -c gradle shadowJar" did not complete successfully: exit code: 1

The error for shippingservice is the following

 => ERROR [shippingservice builder 3/8] RUN if [ "linux/amd64" = "linux/arm64" ] ; then  apt-get update && apt-get install --no-install-recommends -y g++ libc6-dev libprotobuf-dev protobuf-comp  0.1s
------
 > [shippingservice builder 3/8] RUN if [ "linux/amd64" = "linux/arm64" ] ; then         apt-get update && apt-get install --no-install-recommends -y g++ libc6-dev libprotobuf-dev protobuf-compiler ca-certificates;     elif [ "linux/amd64" = "linux/arm64" ] ; then         apt-get update && apt-get install --no-install-recommends -y g++-aarch64-linux-gnu libc6-dev-arm64-cross libprotobuf-dev protobuf-compiler ca-certificates &&         rustup target add aarch64-unknown-linux-gnu &&         rustup toolchain install stable-aarch64-unknown-linux-gnu;     else         echo "linux/amd64 is not supported";         exit 1;     fi:
0.113 linux/amd64 is not supported
------
failed to solve: process "/bin/sh -c if [ \"${TARGETPLATFORM}\" = \"${BUILDPLATFORM}\" ] ; then         apt-get update && apt-get install --no-install-recommends -y g++ libc6-dev libprotobuf-dev protobuf-compiler ca-certificates;     elif [ \"${TARGETPLATFORM}\" = \"linux/arm64\" ] ; then         apt-get update && apt-get install --no-install-recommends -y g++-aarch64-linux-gnu libc6-dev-arm64-cross libprotobuf-dev protobuf-compiler ca-certificates &&         rustup target add aarch64-unknown-linux-gnu &&         rustup toolchain install stable-aarch64-unknown-linux-gnu;     else         echo \"${TARGETPLATFORM} is not supported\";         exit 1;     fi" did not complete successfully: exit code: 1

I'm not an expert in cross-compilation with Docker and i'm pretty sure i'm missing something here. Thanks for your help.

[julianocosta89]

What happens if you run docker compose build without any build args?

[camsellem]

It will complete the build but images will be in arm64 format which is my native platform.

[julianocosta89]

Not quite following the issue then 😅
Could elaborate on the issue @camsellem ?

[camsellem]

Yes - sorry if it wasn't clear. If I compile the images locally on my Macbook M3 it will use the native architecture which is arm64.

My goal is to compile and run the images locally (using Docker emulation mode) for the linux/amd64 so when I deploy my images on GCP/GKE it works natively.

In Docker using the --platform flag does provide the target architecture during the build phase. But when I set it to linux/amd64 I get the above errors for two services. For all other services I'm able to build the images for linux/amd64 arch and run it locally in Docker via emulation.

[julianocosta89]

And if you build without the --platform, aren't you able to run it locally?
I'm confused because I've never used the --platform before.

I believe we have added that param so the Github Action could set it dynamically during build time.

[puckpuck]

This should be fixed in #1651

[camsellem]

Thanks @puckpuck that;s what I was after