Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support cross compiling to BSD and CI it #8887

Merged
merged 6 commits into from
Sep 25, 2023
Merged

Support cross compiling to BSD and CI it #8887

merged 6 commits into from
Sep 25, 2023

Conversation

Ericson2314
Copy link
Member

Motivation

Fix #3280

Thanks to @alyssais effort, BSD cross compilation in Nixpkgs is now quite mature. Let's take advantage of this with Nix itself --- we've accepted patches to allow BSD builds before, but without CI bitrotting was quite likely. Now it won't bitrot, and people down download pre-built binaries.

Context

depends on #8569

Checklist for maintainers

Maintainers: tick if completed or explain if not relevant

  • agreed on idea
  • agreed on implementation strategy
  • tests, as appropriate
    • functional tests - tests/**.sh
    • unit tests - src/*/tests
    • integration tests - tests/nixos/*
  • documentation in the manual
  • documentation in the internal API docs
  • code and comments are self-explanatory
  • commit message explains why the change was made
  • new feature or incompatible change: updated release notes

Priorities

Add 👍 to pull requests you find important.

@github-actions github-actions bot added the with-tests Issues related to testing. PRs with tests have some priority label Aug 31, 2023
@Ericson2314 Ericson2314 changed the title Support cross compile to BSD and CI it Support cross compiling to BSD and CI it Aug 31, 2023
@Ericson2314
Copy link
Member Author

Ericson2314 commented Aug 31, 2023 •
edited
Loading

No idea what is going on with nix flake check, hopefully it is easy to fix?

@roberth
Copy link
Member

roberth commented Sep 1, 2023

hopefully it is easy to fix?

Maybe crossSystem is null? We should get rid of that thing, because we don't need to use nullable values to represent compilation platform settings. How about just hostPlatform and buildPlatform? That's a nice product type that should clean up the code nicely.

@edolstra edolstra added the idea approved The given proposal has been discussed and approved by the Nix team. An implementation is welcome. label Sep 1, 2023
@roberth
Copy link
Member

roberth commented Sep 1, 2023

Meeting notes 2023-09-01

  • Surprise by the small code patch.
  • @edolstra: Do we guarantee support for these new (cross) builds? Should it block a release?
  • @roberth: Let's make it best-effort.
  • @roberth: We could support without risk by building it in pull request CI.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-09-01-nix-team-meeting-minutes-84/32466/1

@Ericson2314 Ericson2314 mentioned this pull request Sep 2, 2023
Merged
8 tasks
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/is-nixbsd-a-posibility/29612/33

@Ericson2314 Ericson2314 mentioned this pull request Sep 12, 2023
Closed
13 tasks
@dpulls
Copy link

dpulls bot commented Sep 20, 2023

🎉 All dependencies have been resolved !

@Ericson2314
Add FreeBSD and NetBSD cross to Nix's flake
e44d2a6
@Ericson2314
Make dev shells work for cross
28850ee 
Need to get tools from right package set. Could build clang tools but I
don't want to wait :D.
@Ericson2314
Make libsodium an unconditional dependency
564392b 
The configure script will not tolerate it being missing.
@Ericson2314
Do not build docs in cross devShell
0db251e 
Coppied from the main build; we really should deduplicate this more.
@Ericson2314
Rename an identifier of ours called stdout
7f76d7f 
This is a reserved identifier on NetBSD --- it is replaced by a macro on
that platform --- and so we cannot use it.
@Ericson2314
Fix boehmgc-coroutine-sp-fallback.diff for FreeBSD
c189116 
Our FreeBSD headers have `pthread_getattr_np`, but we get a link-time
error that is missing. The good news is that there is another similar
function which does exist, and the upstream project elsewhere does just
the [fallback code] we need.

As the fallback code indicates, the two functions are not identical
however as the other one needs explicit initialization. NetBSD supports
both in fact, and its [manpage] is therefore a good
resource on what the differences are.

[fallback code]: https://github.com/ivmai/bdwgc/blob/07a6d0ee8889bca5eaeadc13cabadc363725d216/os_dep.c#L1266-L1272

[manpage]: https://man.netbsd.org/pthread_attr_get_np.3
@github-actions github-actions bot removed the with-tests Issues related to testing. PRs with tests have some priority label Sep 20, 2023
roberth
roberth reviewed Sep 25, 2023
View reviewed changes
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A question. Otherwise lgtm.

flake.nix
++ lib.optional stdenv.cc.isClang pkgs.buildPackages.bear
++ lib.optional
(stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform)
pkgs.buildPackages.clang-tools
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for this? buildPackages isn't the weird package set.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that might actually be fine. It was just doing a really slow build and I did not want to wait. (Because Nixpkgs regressed and now has a target specific LLVM build for no good reasons because the way it depends on binutils rrather than libbfd for LTO.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that in a follow-up PR,

@Ericson2314 Ericson2314 merged commit 1f3fc08 into NixOS:master Sep 25, 2023
8 checks passed
@Ericson2314 Ericson2314 deleted the bsd-cross-ci branch September 25, 2023 17:46
@roberth roberth mentioned this pull request Oct 6, 2023
Open
lf- added a commit to lix-project/lix that referenced this pull request May 9, 2024
@lf-
fix: eval error of .#devShells.x86_64-linux.x86_64-freebsd13
d1dacad 
This is broken and our resident nixbsd maintainers say it should
probably just be temporarily removed till we switch to 24.05 instead of
diagnosing it.

Originally introduced in: NixOS/nix#8887

Fixes: https://git.lix.systems/lix-project/lix/issues/277
Change-Id: I1e7db8859620024a7b37dbd0cc1c5ec139b9e5cb
@Ericson2314 Ericson2314 added the backport 2.18-maintenance Automatically creates a PR against the branch label Jun 20, 2024
@github-actions GitHub Actions
Copy link

Backport failed for 2.18-maintenance, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.18-maintenance
git worktree add -d .worktree/backport-8887-to-2.18-maintenance origin/2.18-maintenance
cd .worktree/backport-8887-to-2.18-maintenance
git switch --create backport-8887-to-2.18-maintenance
git cherry-pick -x e44d2a6bbef113b3d8f162f75bd5c94e0101075f 28850ee90095e337bf47fbe03a0d937cb98784e2 564392b57bc44c407303e4eaf6138d61a8ea50b0 0db251e4ad07338375fd59134fb467e9ebc4176a 7f76d7f038fbb5cb7982cf9aa951b9730566e275 c18911602eb4260d59acf8c17f1c3b4c7fcf7cee

tebowy pushed a commit to tebowy/nix that referenced this pull request Jul 11, 2024
@Ericson2314
Merge pull request NixOS#8887 from obsidiansystems/bsd-cross-ci
bc4aa3d 
Support cross compiling to BSD and CI it

(cherry picked from commit 1f3fc08)
Change-Id: I415e92952afc661cfb5ef91a76c0637678a04a19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@roberth roberth roberth left review comments

@edolstra edolstra Awaiting requested review from edolstra edolstra is a code owner

Assignees
No one assigned
Labels
backport 2.18-maintenance Automatically creates a PR against the branch idea approved The given proposal has been discussed and approved by the Nix team. An implementation is welcome.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add NetBSD/FreeBSD Support
4 participants
@Ericson2314 @roberth @nixos-discourse @edolstra