Ghc cross (#2)

* haskellPackages: stackage LTS 22.31 -> LTS 22.33 This commit has been generated by maintainers/scripts/haskell/update-stackage.sh * all-cabal-hashes: 2024-07-31T18:11:52Z -> 2024-08-19T17:17:03Z This commit has been generated by maintainers/scripts/haskell/update-hackage.sh * haskellPackages: regenerate package set based on current config This commit has been generated by maintainers/scripts/haskell/regenerate-hackage-packages.sh * haskellPackages.Cabal_3_{10_3,12_1}_0: use process 1.6.22.0 * haskellPackages.feedback: use safe-coloured-text-layout 0.2.0.1 * haskellPackages.entropy: build with js backend * haskell.compiler.ghc*: set stage 0 tools This is easy in comparison since these tools won't end up in GHC's settings nor need to be available at runtime, so we can use the *_FOR_BUILD environment variables. It is important to add buildCC to depsBuildBuild to engage the stdenv/wrapper script machinery properly. Co-authored-by: sternenseemann <[email protected]> * haskell.compiler.ghc*: fix libffi cross dependency * haskell.compiler.ghc*: correctly account for ncurses in cross 1. Explicitly set WITH_TERMINFO. We usually match GHC's behavior well, but it is better to tie the Nix option to make explicitly. Unfortunately, the same is very complicated to achieve with hadrian (iirc). 2. Disable enableTerminfo if we are cross-compiling. This matches the behavior of GHC's build system, so we'll have to match it now. It also reduces the ncurses-related headache a bit. 3. Stop passing --with-curses* flags. Unfortunately, GHC does not account for the fact that different platforms need different ncurses libraries. This is somewhat migitated by the fact that ncurses is only ever needed for the build platform if we are cross compiling, but I seem to remember it leaking into the final GHC somehow. A more reliable alternative is relying on the cc/ld wrapper scripts, as they'll always pull out the correct ncurses out of the environment when GHC's build system passes -lcurses. 4. Unconditionally add ncurses to depsBuildBuild. Stage0 unconditionally builds terminfo (maybe the stage1 compiler needs it?), so we need to make sure that ncurses for the build platform is available. Co-authored-by: sternenseemann <[email protected]> * haskell.compiler.ghc*: don't configure --host * haskell.compiler.ghc*: use target CC in settings * haskell.compiler.ghc*: don't cross-compile haddock * haskell.compiler.ghc*: add enableUnregisterised * haskell.compiler.ghc*: assert cross is possible * haskell.compiler.ghc*: allow all platforms Propagating the supported platforms of the boot compiler doesn't make much sense when unregistered cross-compilation is possible. * haskell.compiler.ghc*: use GHC from pkgsBuildBuild * tests.cross.sanity: Add GHC to catch regressions This is to ensure that Haskell users on platforms that lack official bindists still have a convenient means of getting GHC running natively. In my admittedly somewhat limited testing on RISC-V, GHC 8.10.7 is able to bootstrap native builds for 9.2.8 and 9.4.5. GHC 9.2.8 and 9.4.5 are unable to bootstrap themselves and 9.6.2 when cross-compiled. If you're looking at this commit to see whether you can safely upgrade the compiler used here to remove 8.10, please try cross-compiling 9.0 or later and then booting a native GHC with it. * ghc-settings-edit: init at 0.1.0 * !fixup haskell.compiler.ghc948: use ghc-settings-edit * fixup! ghc-settings-edit * haskell.compile.ghc{96*,98*,910*}: fix libdw cross dependency * !fixup make/native-bignum: use toolPath --------- Co-authored-by: maralorn <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: sternenseemann <[email protected]> Co-authored-by: Alexandre Esteves <[email protected]> Co-authored-by: Alex Tunstall <[email protected]>
bolives-hax · Sep 3, 2024 · 9c02e9a · 9c02e9a
1 parent c4803ca
commit 9c02e9a
Show file tree

Hide file tree

Showing 13 changed files with 1,953 additions and 776 deletions.
diff --git a/pkgs/data/misc/hackage/pin.json b/pkgs/data/misc/hackage/pin.json
@@ -1,6 +1,6 @@
 {
-  "commit": "32e512a9cd79f962a8023e1c8c1a4db17d9b2fd8",
-  "url": "https://github.com/commercialhaskell/all-cabal-hashes/archive/32e512a9cd79f962a8023e1c8c1a4db17d9b2fd8.tar.gz",
-  "sha256": "1wiy2fsz8a1gx6mbzq8ns6wvns6kh2aigxczw0f9q7qbkd22kdil",
-  "msg": "Update from Hackage at 2024-07-31T18:11:52Z"
+  "commit": "4670ba3f52eae6737137ddcb817182f3daad4f26",
+  "url": "https://github.com/commercialhaskell/all-cabal-hashes/archive/4670ba3f52eae6737137ddcb817182f3daad4f26.tar.gz",
+  "sha256": "18q5shdmh12147502135q6132rmzp29qfjhwrcjwidl0pi9ki70i",
+  "msg": "Update from Hackage at 2024-08-19T17:17:03Z"
 }
diff --git a/pkgs/development/compilers/ghc/8.10.7.nix b/pkgs/development/compilers/ghc/8.10.7.nix
@@ -1,4 +1,4 @@
-{ lib, stdenv, pkgsBuildTarget, pkgsHostTarget, targetPackages
+{ lib, stdenv, pkgsBuildTarget, pkgsHostTarget, buildPackages, targetPackages
 
 # build-tools
 , bootPkgs
@@ -39,7 +39,10 @@
   enableShared ? !stdenv.targetPlatform.isWindows && !stdenv.targetPlatform.useiOSPrebuilt
 
 , # Whether to build terminfo.
-  enableTerminfo ? !stdenv.targetPlatform.isWindows
+  enableTerminfo ? !(stdenv.targetPlatform.isWindows
+                     # terminfo can't be built for cross
+                     || (stdenv.buildPlatform != stdenv.hostPlatform)
+                     || (stdenv.hostPlatform != stdenv.targetPlatform))
 
 , # What flavour to build. An empty string indicates no
   # specific flavour and falls back to ghc default values.
@@ -48,28 +51,37 @@
 
 , #  Whether to build sphinx documentation.
   enableDocs ? (
-    # Docs disabled if we are building on musl because it's a large task to keep
-    # all `sphinx` dependencies building in this environment.
-    !stdenv.buildPlatform.isMusl
+    # Docs disabled if we are building on musl or cross-building because it's a
+    # large task to keep all `sphinx` dependencies building in this environment.
+    (stdenv.buildPlatform == stdenv.hostPlatform && stdenv.targetPlatform == stdenv.hostPlatform)
+    && !stdenv.buildPlatform.isMusl
   )
 
 , enableHaddockProgram ?
     # Disabled for cross; see note [HADDOCK_DOCS].
-    (stdenv.targetPlatform == stdenv.hostPlatform)
+    (stdenv.buildPlatform == stdenv.hostPlatform && stdenv.targetPlatform == stdenv.hostPlatform)
 
 , # Whether to disable the large address space allocator
   # necessary fix for iOS: https://www.reddit.com/r/haskell/comments/4ttdz1/building_an_osxi386_to_iosarm64_cross_compiler/d5qvd67/
   disableLargeAddressSpace ? stdenv.targetPlatform.isiOS
+
+, # Whether to build an unregisterised version of GHC.
+  # GHC will normally auto-detect whether it can do a registered build, but this
+  # option will force it to do an unregistered build when set to true.
+  # See https://gitlab.haskell.org/ghc/ghc/-/wikis/building/unregisterised
+  enableUnregisterised ? false
 }@args:
 
 assert !enableIntegerSimple -> gmp != null;
 
 # Cross cannot currently build the `haddock` program for silly reasons,
 # see note [HADDOCK_DOCS].
-assert (stdenv.targetPlatform != stdenv.hostPlatform) -> !enableHaddockProgram;
+assert (stdenv.buildPlatform != stdenv.hostPlatform || stdenv.targetPlatform != stdenv.hostPlatform) -> !enableHaddockProgram;
+
+# GHC does not support building when all 3 platforms are different.
+assert stdenv.buildPlatform == stdenv.hostPlatform || stdenv.hostPlatform == stdenv.targetPlatform;
 
 let
-  libffi_name = if stdenv.isDarwin && stdenv.isAarch64 then "libffi" else "libffi_3_3";
   inherit (stdenv) buildPlatform hostPlatform targetPlatform;
 
   inherit (bootPkgs) ghc;
@@ -86,6 +98,8 @@ let
     endif
     BUILD_SPHINX_HTML = ${if enableDocs then "YES" else "NO"}
     BUILD_SPHINX_PDF = NO
+
+    WITH_TERMINFO = ${if enableTerminfo then "YES" else "NO"}
   '' +
   # Note [HADDOCK_DOCS]:
   # Unfortunately currently `HADDOCK_DOCS` controls both whether the `haddock`
@@ -134,7 +148,9 @@ let
     pkgsBuildTarget.targetPackages.stdenv.cc
   ] ++ lib.optional useLLVM buildTargetLlvmPackages.llvm;
 
+  buildCC = buildPackages.stdenv.cc;
   targetCC = builtins.head toolsForTarget;
+  installCC = pkgsHostTarget.targetPackages.stdenv.cc;
 
   # toolPath calculates the absolute path to the name tool associated with a
   # given `stdenv.cc` derivation, i.e. it picks the correct derivation to take
@@ -185,14 +201,12 @@ let
     (lib.optionalString enableIntegerSimple "-integer-simple")
   ];
 
-in
+  libffi_name = if stdenv.isDarwin && stdenv.isAarch64 then "libffi" else "libffi_3_3";
+  targetLibffi = if hostPlatform != targetPlatform
+    then targetPackages.${libffi_name}
+    else pkgsHostTarget.${libffi_name};
 
-# C compiler, bintools and LLVM are used at build time, but will also leak into
-# the resulting GHC's settings file and used at runtime. This means that we are
-# currently only able to build GHC if hostPlatform == buildPlatform.
-assert targetCC == pkgsHostTarget.targetPackages.stdenv.cc;
-assert buildTargetLlvmPackages.llvm == llvmPackages.llvm;
-assert stdenv.targetPlatform.isDarwin -> buildTargetLlvmPackages.clang == llvmPackages.clang;
+in
 
 stdenv.mkDerivation (rec {
   version = "8.10.7";
@@ -299,6 +313,10 @@ stdenv.mkDerivation (rec {
     # LLVM backend on Darwin needs clang: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/codegens.html#llvm-code-generator-fllvm
     export CLANG="${buildTargetLlvmPackages.clang}/bin/${buildTargetLlvmPackages.clang.targetPrefix}clang"
   '' + ''
+    # No need for absolute paths since these tools only need to work during the build
+    export CC_STAGE0="$CC_FOR_BUILD"
+    export LD_STAGE0="$LD_FOR_BUILD"
+    export AR_STAGE0="$AR_FOR_BUILD"
 
     echo -n "${buildMK}" > mk/build.mk
     sed -i -e 's|-isysroot /Developer/SDKs/MacOSX10.5.sdk||' configure
@@ -329,18 +347,20 @@ stdenv.mkDerivation (rec {
       done
   '';
 
+  # Although it is usually correct to pass --host, we don't do that here because
+  # GHC's usage of build, host, and target is non-standard.
+  # See https://gitlab.haskell.org/ghc/ghc/-/wikis/building/cross-compiling
   # TODO(@Ericson2314): Always pass "--target" and always prefix.
-  configurePlatforms = [ "build" "host" ]
-    ++ lib.optional (targetPlatform != hostPlatform) "target";
+  configurePlatforms = [ "build" ]
+    ++ lib.optional (buildPlatform != hostPlatform || targetPlatform != hostPlatform) "target";
 
   # `--with` flags for libraries needed for RTS linker
   configureFlags = [
     "--datadir=$doc/share/doc/ghc"
-    "--with-curses-includes=${ncurses.dev}/include" "--with-curses-libraries=${ncurses.out}/lib"
   ] ++ lib.optionals (args.${libffi_name} != null) [
     "--with-system-libffi"
-    "--with-ffi-includes=${targetPackages.${libffi_name}.dev}/include"
-    "--with-ffi-libraries=${targetPackages.${libffi_name}.out}/lib"
+    "--with-ffi-includes=${targetLibffi.dev}/include"
+    "--with-ffi-libraries=${targetLibffi.out}/lib"
   ] ++ lib.optionals (targetPlatform == hostPlatform && !enableIntegerSimple) [
     "--with-gmp-includes=${targetPackages.gmp.dev}/include"
     "--with-gmp-libraries=${targetPackages.gmp.out}/lib"
@@ -355,6 +375,8 @@ stdenv.mkDerivation (rec {
     "CONF_GCC_LINKER_OPTS_STAGE2=-fuse-ld=gold"
   ] ++ lib.optionals (disableLargeAddressSpace) [
     "--disable-large-address-space"
+  ] ++ lib.optionals enableUnregisterised [
+    "--enable-unregisterised"
   ];
 
   # Make sure we never relax`$PATH` and hooks support for compatibility.
@@ -372,6 +394,12 @@ stdenv.mkDerivation (rec {
     sphinx
   ];
 
+  # Used by the STAGE0 compiler to build stage1
+  depsBuildBuild = [
+    buildCC
+    # stage0 builds terminfo unconditionally, so we always need ncurses
+    ncurses
+  ];
   # For building runtime libs
   depsBuildTarget = toolsForTarget;
 
@@ -402,6 +430,14 @@ stdenv.mkDerivation (rec {
   requiredSystemFeatures = [ "big-parallel" ];
 
   postInstall = ''
+    # Make the installed GHC use the host platform's tools.
+    sed -i $out/lib/${targetPrefix}${passthru.haskellCompilerName}/settings \
+      -e "s!$CC!${installCC}/bin/${installCC.targetPrefix}cc!g" \
+      -e "s!$CXX!${installCC}/bin/${installCC.targetPrefix}c++!g" \
+      -e "s!$LD!${installCC.bintools}/bin/${installCC.bintools.targetPrefix}ld${lib.optionalString useLdGold ".gold"}!g" \
+      -e "s!$AR!${installCC.bintools.bintools}/bin/${installCC.bintools.targetPrefix}ar!g" \
+      -e "s!$RANLIB!${installCC.bintools.bintools}/bin/${installCC.bintools.targetPrefix}ranlib!g"
+
     # Install the bash completion file.
     install -D -m 444 utils/completion/ghc.bash $out/share/bash-completion/completions/${targetPrefix}ghc
   '';
@@ -427,7 +463,8 @@ stdenv.mkDerivation (rec {
       guibou
     ] ++ lib.teams.haskell.members;
     timeout = 24 * 3600;
-    inherit (ghc.meta) license platforms;
+    platforms = lib.platforms.all;
+    inherit (ghc.meta) license;
   };
 
 } // lib.optionalAttrs targetPlatform.useAndroidPrebuilt {

diff --git a/pkgs/development/compilers/ghc/common-hadrian.nix b/pkgs/development/compilers/ghc/common-hadrian.nix
@@ -12,6 +12,7 @@
 , stdenv
 , pkgsBuildTarget
 , pkgsHostTarget
+, buildPackages
 , targetPackages
 , fetchpatch
 
@@ -65,8 +66,14 @@
   enableShared ? with stdenv.targetPlatform; !isWindows && !useiOSPrebuilt && !isStatic && !isGhcjs
 
 , # Whether to build terminfo.
+  # FIXME(@sternenseemann): This actually doesn't influence what hadrian does,
+  # just what buildInputs etc. looks like. It would be best if we could actually
+  # tell it what to do like it was possible with make.
   enableTerminfo ? !(stdenv.targetPlatform.isWindows
-                     || stdenv.targetPlatform.isGhcjs)
+                     || stdenv.targetPlatform.isGhcjs
+                     # terminfo can't be built for cross
+                     || (stdenv.buildPlatform != stdenv.hostPlatform)
+                     || (stdenv.hostPlatform != stdenv.targetPlatform))
 
 , # Libdw.c only supports x86_64, i686 and s390x as of 2022-08-04
   enableDwarf ? (stdenv.targetPlatform.isx86 ||
@@ -214,18 +221,31 @@
 , #  Whether to build sphinx documentation.
   # TODO(@sternenseemann): Hadrian ignores the --docs flag if finalStage = Stage1
   enableDocs ? (
-    # Docs disabled if we are building on musl because it's a large task to keep
-    # all `sphinx` dependencies building in this environment.
-    !stdenv.buildPlatform.isMusl
+    # Docs disabled if we are building on musl or cross-building because it's a
+    # large task to keep all `sphinx` dependencies building in this environment.
+    (stdenv.buildPlatform == stdenv.hostPlatform && stdenv.targetPlatform == stdenv.hostPlatform)
+    && !stdenv.buildPlatform.isMusl
   )
 
 , # Whether to disable the large address space allocator
   # necessary fix for iOS: https://www.reddit.com/r/haskell/comments/4ttdz1/building_an_osxi386_to_iosarm64_cross_compiler/d5qvd67/
   disableLargeAddressSpace ? stdenv.targetPlatform.isiOS
+
+, # Whether to build an unregisterised version of GHC.
+  # GHC will normally auto-detect whether it can do a registered build, but this
+  # option will force it to do an unregistered build when set to true.
+  # See https://gitlab.haskell.org/ghc/ghc/-/wikis/building/unregisterised
+  enableUnregisterised ? false
 }:
 
 assert !enableNativeBignum -> gmp != null;
 
+# GHC does not support building when all 3 platforms are different.
+assert stdenv.buildPlatform == stdenv.hostPlatform || stdenv.hostPlatform == stdenv.targetPlatform;
+
+# It is currently impossible to cross-compile GHC with Hadrian.
+assert stdenv.buildPlatform == stdenv.hostPlatform;
+
 let
   inherit (stdenv) buildPlatform hostPlatform targetPlatform;
 
@@ -265,7 +285,9 @@ let
      else pkgsBuildTarget.targetPackages.stdenv.cc)
   ] ++ lib.optional useLLVM buildTargetLlvmPackages.llvm;
 
+  buildCC = buildPackages.stdenv.cc;
   targetCC = builtins.head toolsForTarget;
+  installCC = pkgsHostTarget.targetPackages.stdenv.cc;
 
   # toolPath calculates the absolute path to the name tool associated with a
   # given `stdenv.cc` derivation, i.e. it picks the correct derivation to take
@@ -316,6 +338,17 @@ let
     (lib.optionalString enableNativeBignum "-native-bignum")
   ];
 
+  targetLibffi = if hostPlatform != targetPlatform
+    then targetPackages.libffi
+    else pkgsHostTarget.libffi;
+
+  targetElfutils = if hostPlatform != targetPlatform
+    then targetPackages.elfutils
+    else pkgsHostTarget.elfutils;
+
+  # Our Cabal compiler name
+  haskellCompilerName = "ghc-${version}";
+
 in
 
 # C compiler, bintools and LLVM are used at build time, but will also leak into
@@ -346,6 +379,10 @@ stdenv.mkDerivation ({
     for env in $(env | grep '^TARGET_' | sed -E 's|\+?=.*||'); do
       export "''${env#TARGET_}=''${!env}"
     done
+    # No need for absolute paths since these tools only need to work during the build
+    export CC_STAGE0="$CC_FOR_BUILD"
+    export LD_STAGE0="$LD_FOR_BUILD"
+    export AR_STAGE0="$AR_FOR_BUILD"
     # GHC is a bit confused on its cross terminology, as these would normally be
     # the *host* tools.
     export CC="${toolPath "cc" targetCC}"
@@ -433,11 +470,10 @@ stdenv.mkDerivation ({
   # `--with` flags for libraries needed for RTS linker
   configureFlags = [
     "--datadir=$doc/share/doc/ghc"
-    "--with-curses-includes=${ncurses.dev}/include" "--with-curses-libraries=${ncurses.out}/lib"
   ] ++ lib.optionals (libffi != null && !targetPlatform.isGhcjs) [
     "--with-system-libffi"
-    "--with-ffi-includes=${targetPackages.libffi.dev}/include"
-    "--with-ffi-libraries=${targetPackages.libffi.out}/lib"
+    "--with-ffi-includes=${targetLibffi.dev}/include"
+    "--with-ffi-libraries=${targetLibffi.out}/lib"
   ] ++ lib.optionals (targetPlatform == hostPlatform && !enableNativeBignum) [
     "--with-gmp-includes=${targetPackages.gmp.dev}/include"
     "--with-gmp-libraries=${targetPackages.gmp.out}/lib"
@@ -454,14 +490,16 @@ stdenv.mkDerivation ({
     "--disable-large-address-space"
   ] ++ lib.optionals enableDwarf [
     "--enable-dwarf-unwind"
-    "--with-libdw-includes=${lib.getDev targetPackages.elfutils}/include"
-    "--with-libdw-libraries=${lib.getLib targetPackages.elfutils}/lib"
+    "--with-libdw-includes=${lib.getDev targetElfutils}/include"
+    "--with-libdw-libraries=${lib.getLib targetElfutils}/lib"
   ] ++ lib.optionals targetPlatform.isDarwin [
     # Darwin uses llvm-ar. GHC will try to use `-L` with `ar` when it is `llvm-ar`
     # but it doesn’t currently work because Cabal never uses `-L` on Darwin. See:
     # https://gitlab.haskell.org/ghc/ghc/-/issues/23188
     # https://github.com/haskell/cabal/issues/8882
     "fp_cv_prog_ar_supports_dash_l=no"
+  ] ++ lib.optionals enableUnregisterised [
+    "--enable-unregisterised"
   ];
 
   # Make sure we never relax`$PATH` and hooks support for compatibility.
@@ -484,6 +522,12 @@ stdenv.mkDerivation ({
 
   # For building runtime libs
   depsBuildTarget = toolsForTarget;
+  # Used by the STAGE0 compiler to build stage1
+  depsBuildBuild = [
+    buildCC
+    # stage0 builds terminfo unconditionally, so we always need ncurses
+    ncurses
+  ];
 
   buildInputs = [ perl bash ] ++ (libDeps hostPlatform);
 
@@ -553,19 +597,24 @@ stdenv.mkDerivation ({
     # leave bindist directory
     popd
 
+    # Make the installed GHC use the host platform's tools.
+    sed -i $out/lib/${targetPrefix}${haskellCompilerName}/lib/settings \
+      -e "s!$CC!${installCC}/bin/${installCC.targetPrefix}cc!g" \
+      -e "s!$CXX!${installCC}/bin/${installCC.targetPrefix}c++!g" \
+      -e "s!$LD!${installCC.bintools}/bin/${installCC.bintools.targetPrefix}ld${lib.optionalString useLdGold ".gold"}!g" \
+      -e "s!$AR!${installCC.bintools.bintools}/bin/${installCC.bintools.targetPrefix}ar!g" \
+      -e "s!$RANLIB!${installCC.bintools.bintools}/bin/${installCC.bintools.targetPrefix}ranlib!g"
+
     # Install the bash completion file.
     install -Dm 644 utils/completion/ghc.bash $out/share/bash-completion/completions/${targetPrefix}ghc
   '';
 
   passthru = {
-    inherit bootPkgs targetPrefix;
+    inherit bootPkgs targetPrefix haskellCompilerName;
 
     inherit llvmPackages;
     inherit enableShared;
 
-    # Our Cabal compiler name
-    haskellCompilerName = "ghc-${version}";
-
     # Expose hadrian used for bootstrapping, for debugging purposes
     inherit hadrian;
 
@@ -581,7 +630,8 @@ stdenv.mkDerivation ({
       guibou
     ] ++ lib.teams.haskell.members;
     timeout = 24 * 3600;
-    inherit (ghc.meta) license platforms;
+    platforms = lib.platforms.all;
+    inherit (ghc.meta) license;
   };
 
   dontStrip = targetPlatform.useAndroidPrebuilt || targetPlatform.isWasm;