From a65999d2542d2f3d63dd26cd6590d16e076bc7c1 Mon Sep 17 00:00:00 2001 From: Sukant Hajra Date: Sat, 13 Jul 2024 22:13:40 -0500 Subject: [PATCH] Docs: Garnix + Nix 2.18 recommendation --- .github/workflows/ci.yml | 3 + doc/internal/links.org | 10 +-- doc/internal/params.el | 3 +- doc/nix-installation.md | 97 +++++++++++++++++-------- doc/nix-installation.org | 153 ++++++++++++++++++++++++--------------- doc/nix-introduction.md | 13 +++- doc/nix-introduction.org | 57 ++++++++++++--- 7 files changed, 228 insertions(+), 108 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b84b25a..4bc9d5b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -12,6 +12,9 @@ jobs: steps: - uses: actions/checkout@v4 - uses: cachix/install-nix-action@v27 + with: + # DESIGN: matching current in NixOS 24.05 + install_url: https://releases.nixos.org/nix/nix-2.18.2/install - uses: cachix/cachix-action@v15 with: name: shajra diff --git a/doc/internal/links.org b/doc/internal/links.org index 0354fd8..b559b2f 100644 --- a/doc/internal/links.org +++ b/doc/internal/links.org @@ -1,9 +1,9 @@ #+link: cachix https://cachix.org #+link: docstring https://en.wikipedia.org/wiki/Docstring #+link: fhs https://www.pathname.com/fhs/ -#+link: garnix https://garnix.io/docs/caching +#+link: garnix https://garnix.io +#+link: garnix-config https://garnix.io/docs/caching #+link: gfm https://github.github.com/gfm/ -#+link: github-actions https://github.com/shajra/nix-project/actions #+link: litprog https://en.wikipedia.org/wiki/Literate_programming #+link: nix https://nixos.org/nix #+link: nix-flake https://nixos.wiki/wiki/Flakes @@ -13,17 +13,17 @@ #+link: nix-flake-parts https://github.com/hercules-ci/flake-parts #+link: nix-flake-parts-doc https://flake.parts #+link: nix-flake-parts-doc-modargs https://flake.parts/module-arguments.html -#+link: nix-install-manual https://nixos.org/manual/nix/stable/installation/installation.html -#+link: nix-install-quick https://nixos.org/download.html#download-nix +#+link: nix-install-manual https://nixos.org/manual/nix/stable/installation/installation #+link: nix-language-manual https://nixos.org/manual/nix/stable/language/index.html #+link: nix-language-tutorial https://nixos.org/guides/nix-language.html #+link: nix-learn https://nixos.org/learn.html #+link: nixos https://nixos.org +#+link: nixos-releases https://nixos.org/manual/nixos/stable/release-notes.html #+link: nixos-search https://search.nixos.org/packages #+link: nixpkgs https://github.com/NixOS/nixpkgs #+link: nixpkgs-manual https://nixos.org/nixpkgs/manual #+link: nixpkgs-manual-langs https://nixos.org/manual/nixpkgs/stable/#chap-language-support -#+link: nix-uninstall https://nixos.org/manual/nix/stable/installation/installing-binary.html#uninstalling +#+link: nix-uninstall https://nixos.org/manual/nix/stable/installation/uninstall #+link: org https://www.gnu.org/software/emacs/manual/html_node/emacs/Org-Mode.html #+link: org-blocks-src https://orgmode.org/manual/Working-with-Source-Code.html#Working-with-Source-Code #+link: org-blocks-src-eval https://orgmode.org/manual/Evaluating-Code-Blocks.html#Evaluating-Code-Blocks diff --git a/doc/internal/params.el b/doc/internal/params.el index eaa5b58..540c554 100644 --- a/doc/internal/params.el +++ b/doc/internal/params.el @@ -10,7 +10,8 @@ ;; run-type must be "executable" (run-target-short . "org2gfm") ;; run-target-long will always prefix "bin/" - (nix-latest . "2.23") + (nix-latest . "2.23.2") + (nix-stable . "2.18.4") (nixos-latest . "24.05") (platforms . "\n\ - Linux on x86-64 machines\n\ diff --git a/doc/nix-installation.md b/doc/nix-installation.md index 4b51d8c..534903b 100644 --- a/doc/nix-installation.md +++ b/doc/nix-installation.md @@ -3,6 +3,9 @@ - [Level of commitment/risk](#sec-3) - [Nix package manager installation](#sec-4) - [Cache setup](#sec-5) + - [System-level cache configuration](#sec-5-1) + - [User-level cache configuration](#sec-5-2) + - [Testing your configuration](#sec-5-3) - [Setting up experimental features](#sec-6) @@ -40,38 +43,83 @@ Hopefully, this alleviates any worry about installing a complex program on your > **NOTE:** You don't need this step if you're running NixOS, which comes with Nix baked in. -If you don't already have Nix, [the official installation script](https://nixos.org/download.html#download-nix) should work on a variety of UNIX-like operating systems. If you're okay with the script calling `sudo` you can install Nix on a non-WSL machine with the following recommended command: +Though the latest version of Nix is Nix 2.23.2, we'll be installing the version that the last release of NixOS (24.05) uses, specifically Nix 2.18.4. As discussed in the included [introduction to Nix](nix-introduction.md), this version is considered stable by the Nix community. + +The following command calls the official installation script for the recommended version of Nix. Note, this script will require `sudo` access. ```bash -sh <(curl -L https://nixos.org/nix/install) --daemon +sh <(curl -L https://releases.nixos.org/nix/nix-2.18.4/install) --daemon ``` The `--daemon` switch installs Nix in the multi-user mode, which is generally recommended (single-user installation with `--no-daemon` instead is recommended for WSL). The script reports everything it does and touches. -After installation, you may have to exit your terminal session and log back in to have environment variables configured to put Nix executables on your `PATH`. +After installation, you may have to exit your terminal session and log back in to have environment variables configured, which puts Nix executables on your `PATH`. + +Every six months or so, a new version of NixOS releases, and you should consider upgrading your installation of Nix. For NixOS 24.05, this command upgrades Nix: + +```bash +NIXOS_VERSION="24.05" +NIX_STORE_PATHS_URL=https://github.com/NixOS/nixpkgs/raw/$NIXOS_VERSION/nixos/module/installer/tools/nix-fallback-paths.nix +sudo nix upgrade-nix --nix-store-paths-url "$NIX_STORE_PATHS_URL" +``` + +For [new releases of NixOS](https://nixos.org/manual/nixos/stable/release-notes.html), use the same command with new `NIXOS_VERSION`. -The Nix manual describes [other methods of installing Nix](https://nixos.org/manual/nix/stable/installation/installation.html) that may suit you more. If you later want to uninstall Nix, see the [uninstallation steps documented in the Nix manual](https://nixos.org/manual/nix/stable/installation/installing-binary.html#uninstalling). +The Nix manual describes [other methods of installing Nix](https://nixos.org/manual/nix/stable/installation/installation) that may suit you more. If you later want to uninstall Nix, see the [uninstallation steps documented in the Nix manual](https://nixos.org/manual/nix/stable/installation/uninstall). # Cache setup -This project pushes built Nix packages to [Cachix](https://cachix.org) as part of its [continuous integration](https://github.com/shajra/nix-project/actions). It's recommended to configure Nix to use shajra.cachix.org as a Nix *substituter*. Once configured, Nix can pull down pre-built packages from Cachix, instead of building them locally (potentially saving time). Cachix will augment Nix's default substituter that pulls from cache.nixos.org. +This project pushes built Nix packages to two substituters, [Garnix](https://garnix.io) and [Cachix](https://cachix.org), as part of its continuous integration. It's recommended to install at least one of these. Configuring both to have a fallback works as well. Garnix caches a few more packages than Cachix. Both should have similar availability. -You can configure shajra.cachix.org as a supplemental substituter with the following command: +We need to extend two settings in either the system-level Nix configuration file at `/etc/nix/nix.conf`, or the user-level configuration at `~/.config/nix/nix.config` (which may not exist yet). -```sh -nix run \ - --file https://cachix.org/api/v1/install \ - cachix \ - --command cachix use shajra -``` +The choice of whether to perform these settings system-level or user-level is up to your preference. + +First we need to specify one or both of the following substituters: + +- +- + +For each substituter we use, we need to also configure Nix to trust their public keys: + +- cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g= +- shajra.cachix.org-1:V0x7Wjgd/mHGk2KQwzXv8iydfIgLupbnZKLSQt5hh9o= + +## System-level cache configuration + +When editing the `/etc/nix/nix.conf` as root, suffix the new substituter(s), space-separated to any values already populating the `substituters` parameter. + +Note, the order of the substituters indicates the order in which caches are searched. Leave the substituter first to maximize cache hits. + +Next, similarly suffix the key(s) to the `trusted-public-keys` parameter. -Cachix is a service that anyone can use. You can call this command later to add substituters for someone else using Cachix, replacing “shajra” with their cache's name. +Your file will likely look like the following: -If you've just run a multi-user Nix installation and are not yet a trusted user in `/etc/nix/nix.conf`, this command may not work. But it will report back some options to proceed. + … + substituters = https://cache.nixos.org/ https://cache.garnix.io https://shajra.cachix.org + trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g= shajra.cachix.org-1:V0x7Wjgd/mHGk2KQwzXv8iydfIgLupbnZKLSQt5hh9o= + … -One option sets you up as a trusted user and installs Cachix configuration for Nix locally at `~/.config/nix/nix.conf`. This configuration will be available immediately, and any subsequent invocation of Nix commands will take advantage of the Cachix cache. +## User-level cache configuration -You can alternatively configure Cachix as a substituter globally by running the above command as a root user (say with `sudo`), which sets up Cachix directly in `/etc/nix/nix.conf`. The invocation may give further instructions upon completion. +User-level Nix configuration overrides system-level settings. For user-level configuration, we can use the `extra-substituters` and `extra-trusted-public-keys` parameters to extend `substituters` and `trusted-public-keys` settings already in the system-level `/etc/nix/nix.conf` file. This way we don't need to worry about accidentally excluding the standard substituter where we get most of our cache hits. + +For user-level Nix configuration, create a file at `~/.config/nix/nix.conf` with the following content: + + … + extra-substituters = https://cache.garnix.io https://shajra.cachix.org + extra-trusted-public-keys = cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g= shajra.cachix.org-1:V0x7Wjgd/mHGk2KQwzXv8iydfIgLupbnZKLSQt5hh9o= + +## Testing your configuration + +You can see that your configuration is correct with the following commands: + +```sh +nix show-config substituters +nix show-config trusted-public-keys +``` + +Make sure still have settings for . # Setting up experimental features @@ -92,20 +140,7 @@ alias nix-flakes = nix --extra-experimental-features 'nix-command flakes' As discussed in the introduction, `nix-command` is enabled by default. You don't need to enable it explicitly (though you could disable it). -To use flakes there are two things we need to do: - -1. make sure the version of Nix we're on is at least 2.4 -2. enable both the `nix-command` and `flakes` experimental features. - -Since the latest release of Nix is already at 2.23, if you installed Nix recently as per the instructions above, you should be on a recent-enough version: - -```sh -nix --version -``` - - nix (Nix) 2.18.2 - -The easiest way to turn on experimental features is to create a file `~/.config/nix/nix.conf` if it doesn't already exist, and in it, put the following line: +The easiest way to turn on experimental features is to put the following setting into either the system-level `/etc/nix/nix.conf` file or the user-level `~/.config/nix/nix.conf` file: ```text experimental-features = nix-command flakes @@ -114,5 +149,5 @@ experimental-features = nix-command flakes Then you should see that the appropriate features are enabled: ```sh -nix show-config | grep experimental-features +nix show-config experimental-features ``` diff --git a/doc/nix-installation.org b/doc/nix-installation.org index a2f1afc..4bc4c3b 100644 --- a/doc/nix-installation.org +++ b/doc/nix-installation.org @@ -18,8 +18,8 @@ following macros and source code blocks (using Noweb). #+macro: get (eval (concat $2 (alist-get (intern $1) (car (read-from-string (f-read "internal/params.el")))) $3)) #+macro: nix-latest {{{get(nix-latest)}}} +#+macro: nix-stable {{{get(nix-stable)}}} #+macro: nixos-latest {{{get(nixos-latest)}}} -#+macro: nixos-branch {{{get(nixos-latest,=nixos-,=)}}} #+macro: platforms {{{get(platforms)}}} * About this document @@ -69,13 +69,16 @@ machine. Uninstallation is not too much more than deleting everything under baked in. #+end_quote -If you don't already have Nix, [[nix-install-quick][the official installation script]] should work on a -variety of UNIX-like operating systems. If you're okay with the script calling -=sudo= you can install Nix on a non-WSL machine with the following recommended -command: +Though the latest version of Nix is Nix {{{nix-latest}}}, we'll be installing +the version that the last release of NixOS ({{{nixos-latest}}}) uses, +specifically Nix {{{nix-stable}}}. As discussed in the included [[file:nix-introduction.org][introduction to +Nix]], this version is considered stable by the Nix community. -#+begin_src bash :eval no -sh <(curl -L https://nixos.org/nix/install) --daemon +The following command calls the official installation script for the recommended +version of Nix. Note, this script will require =sudo= access. + +#+begin_src bash :eval no :noweb yes +sh <(curl -L https://releases.nixos.org/nix/nix-<>/install) --daemon #+end_src The =--daemon= switch installs Nix in the multi-user mode, which is generally @@ -83,7 +86,20 @@ recommended (single-user installation with =--no-daemon= instead is recommended for WSL). The script reports everything it does and touches. After installation, you may have to exit your terminal session and log back in -to have environment variables configured to put Nix executables on your =PATH=. +to have environment variables configured, which puts Nix executables on your +=PATH=. + +Every six months or so, a new version of NixOS releases, and you should consider +upgrading your installation of Nix. For NixOS {{{nixos-latest}}}, this command +upgrades Nix: + +#+begin_src bash :eval no :noweb yes +NIXOS_VERSION="<>" +NIX_STORE_PATHS_URL=https://github.com/NixOS/nixpkgs/raw/$NIXOS_VERSION/nixos/module/installer/tools/nix-fallback-paths.nix +sudo nix upgrade-nix --nix-store-paths-url "$NIX_STORE_PATHS_URL" +#+end_src + +For [[nixos-releases][new releases of NixOS]], use the same command with new =NIXOS_VERSION=. The Nix manual describes [[nix-install-manual][other methods of installing Nix]] that may suit you more. If you later want to uninstall Nix, see the [[nix-uninstall][uninstallation steps documented in @@ -91,39 +107,76 @@ the Nix manual]]. * Cache setup -This project pushes built Nix packages to [[cachix][Cachix]] as part of its [[github-actions][continuous -integration]]. It's recommended to configure Nix to use shajra.cachix.org as a -Nix /substituter/. Once configured, Nix can pull down pre-built packages from -Cachix, instead of building them locally (potentially saving time). Cachix will -augment Nix's default substituter that pulls from cache.nixos.org. +This project pushes built Nix packages to two substituters, [[garnix][Garnix]] and [[cachix][Cachix]], +as part of its continuous integration. It's recommended to install at least one +of these. Configuring both to have a fallback works as well. Garnix caches a few +more packages than Cachix. Both should have similar availability. -You can configure shajra.cachix.org as a supplemental substituter with the -following command: +We need to extend two settings in either the system-level Nix configuration file +at =/etc/nix/nix.conf=, or the user-level configuration at +=~/.config/nix/nix.config= (which may not exist yet). -#+begin_src sh :eval no -nix run \ - --file https://cachix.org/api/v1/install \ - cachix \ - --command cachix use shajra -#+end_src +The choice of whether to perform these settings system-level or user-level is up +to your preference. + +First we need to specify one or both of the following substituters: +- https://cache.garnix.io +- https://shajra.cachix.org + +For each substituter we use, we need to also configure Nix to trust their public +keys: +- cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g= +- shajra.cachix.org-1:V0x7Wjgd/mHGk2KQwzXv8iydfIgLupbnZKLSQt5hh9o= + +** System-level cache configuration + +When editing the =/etc/nix/nix.conf= as root, suffix the new substituter(s), +space-separated to any values already populating the =substituters= parameter. + +Note, the order of the substituters indicates the order in which caches are +searched. Leave the https://cache.nixos.org substituter first to maximize cache +hits. + +Next, similarly suffix the key(s) to the =trusted-public-keys= parameter. -Cachix is a service that anyone can use. You can call this command later to add -substituters for someone else using Cachix, replacing “shajra” with their -cache's name. +Your file will likely look like the following: -If you've just run a multi-user Nix installation and are not yet a trusted user -in =/etc/nix/nix.conf=, this command may not work. But it will report back some -options to proceed. +#+begin_example +… +substituters = https://cache.nixos.org/ https://cache.garnix.io https://shajra.cachix.org +trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g= shajra.cachix.org-1:V0x7Wjgd/mHGk2KQwzXv8iydfIgLupbnZKLSQt5hh9o= +… +#+end_example -One option sets you up as a trusted user and installs Cachix configuration for -Nix locally at =~/.config/nix/nix.conf=. This configuration will be available -immediately, and any subsequent invocation of Nix commands will take advantage -of the Cachix cache. +** User-level cache configuration -You can alternatively configure Cachix as a substituter globally by running the -above command as a root user (say with =sudo=), which sets up Cachix directly in -=/etc/nix/nix.conf=. The invocation may give further instructions upon -completion. +User-level Nix configuration overrides system-level settings. For user-level +configuration, we can use the =extra-substituters= and +=extra-trusted-public-keys= parameters to extend =substituters= and +=trusted-public-keys= settings already in the system-level =/etc/nix/nix.conf= +file. This way we don't need to worry about accidentally excluding the standard +https://cache.nixos.org substituter where we get most of our cache hits. + +For user-level Nix configuration, create a file at =~/.config/nix/nix.conf= with +the following content: + +#+begin_example +… +extra-substituters = https://cache.garnix.io https://shajra.cachix.org +extra-trusted-public-keys = cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g= shajra.cachix.org-1:V0x7Wjgd/mHGk2KQwzXv8iydfIgLupbnZKLSQt5hh9o= +#+end_example + +** Testing your configuration + +You can see that your configuration is correct with the following commands: + +#+name: nix-config-test +#+begin_src sh :eval no +nix show-config substituters +nix show-config trusted-public-keys +#+end_src + +Make sure still have settings for https://cache.nixos.org. * Setting up experimental features @@ -149,28 +202,12 @@ POSIX-compliant shells: alias nix-flakes = nix --extra-experimental-features 'nix-command flakes' #+end_src -As discussed in the introduction, =nix-command= is enabled by default. You -don't need to enable it explicitly (though you could disable it). - -To use flakes there are two things we need to do: -1. make sure the version of Nix we're on is at least 2.4 -2. enable both the =nix-command= and =flakes= experimental features. - -Since the latest release of Nix is already at {{{nix-latest}}}, if you installed -Nix recently as per the instructions above, you should be on a recent-enough -version: - -#+name: nix-version -#+begin_src sh :results output :exports both -nix --version -#+end_src - -#+RESULTS: nix-version -: nix (Nix) 2.18.2 +As discussed in the introduction, =nix-command= is enabled by default. You don't +need to enable it explicitly (though you could disable it). -The easiest way to turn on experimental features is to create a file -=~/.config/nix/nix.conf= if it doesn't already exist, and in it, put the -following line: +The easiest way to turn on experimental features is to put the following setting +into either the system-level =/etc/nix/nix.conf= file or the user-level +=~/.config/nix/nix.conf= file: #+begin_src text :eval no experimental-features = nix-command flakes @@ -180,8 +217,8 @@ Then you should see that the appropriate features are enabled: #+name: nix-show-config #+begin_src sh :results output :export both -nix show-config | grep experimental-features +nix show-config experimental-features #+end_src #+RESULTS: nix-show-config -: experimental-features = flakes nix-command +: flakes nix-command diff --git a/doc/nix-introduction.md b/doc/nix-introduction.md index 7af4f7d..48fe989 100644 --- a/doc/nix-introduction.md +++ b/doc/nix-introduction.md @@ -15,6 +15,7 @@ - [Confusion of stability](#sec-4-2) - [Nix 2.0 and the new `nix` command](#sec-4-2-1) - [Flakes as an experiment](#sec-4-2-2) + - [Nix quick releases compete with stability](#sec-4-2-3) - [A few gaps in determinism](#sec-4-3) - [Encouraging development with flakes](#sec-5) - [Limiting usage of experimental APIs](#sec-5-1) @@ -110,7 +111,7 @@ A system of thorough hashing accomplishes this degree of precision. In Nix, the The repeatability and precision of Nix form the basis of how substituters are trusted as caching services across the world. It also allows us to trust remote builds more easily without worrying about deviations in environment configuration. -Nix has a central substituter at , but there are third-party ones as well, like [Cachix](https://cachix.org) and [Garnix](https://garnix.io/docs/caching). Before building a package, the hash for the package is calculated. If any configured substituter has a build for the hash, it's pulled down as a substitute. A certificate-based protocol is used to establish the trust of substituters. Between this protocol and the algorithm for calculating hashes in Nix, you can have confidence that a package pulled from a substituter will be equivalent to what you would have built locally. +Nix has a central substituter at , but there are third-party ones as well, like [Garnix](https://garnix.io) and [Cachix](https://cachix.org). Before building a package, the hash for the package is calculated. If any configured substituter has a build for the hash, it's pulled down as a substitute. A certificate-based protocol is used to establish the trust of substituters. Between this protocol and the algorithm for calculating hashes in Nix, you can have confidence that a package pulled from a substituter will be equivalent to what you would have built locally. Finally, all packages are stored in `/nix/store` by their hash. This simple scheme allows us to install multiple versions of the same package without conflicts. References to dependencies all point back to the desired version in `/nix/store` they need. Though Nix has not eliminated the risk of concurrently running different versions of the same program, at least the flexibility to do so is in the user's hands. @@ -193,11 +194,19 @@ However, if industrial users move to flakes to address these problems, we have t - we have to be ready for the flakes API to change, as it's technically experimental - we have to accept some added training hurdles since the documentation of flakes is tucked behind documentation of non-flake usage. +### Nix quick releases compete with stability + +The latest major version of the Nix package manager is currently Nix 2.23.2, but NixOS 24.05, the latest stable release of NixOS, uses Nix 2.18.4. NixOS is the primary way the Nix package manager gets used in the field. Far fewer users install Nix as a package manager atop another operating system. From a community perspective it makes sense to consider Nix 2.18.4 the stable release of the package manager. This version gets the most scrutiny and critical bug fixes. + +As mentioned above, there are strong reasons to use still-experimental features, particularly flakes. However, APIs and calculated hashes change too frequently in experimental features from version-to-version. By sticking with the version used in NixOS, we get less breaking changes. For example, the [flake.lock](../flake.lock) file included with this project has calculated hashes for dependencies. These hashes were computed with Nix 2.18.4, and could change with later versions. + +For these reasons, the [installation guide included with this project](nix-installation.md) recommends installing Nix 2.18.4, rather than the latest official release. + ## A few gaps in determinism Nix offers world-class build determinism, especially with flakes. But it's important to understand that this determinism is not infallible. To date, no build system can claim to provide flawless determinism. -Known gaps involve corner cases like the following. Consider a hypothetical compiler that can auto-detect that a build machine has many cores, and enables an optimization upon detection incompatible with machines with fewer cores. While Nix will generate different hashes if the platform architecture changes, say from X86 to ARM, it will not consider a machine with many cores different from one with fewer. So our example optimizing compiler could cause a frustrating problem. A local build on a machine with few cores may work as expected. But if a cache had a optimized build from a machine with many cores, it would be pulled down for the same hash, as a substitute for a local build. This optimization would lead to defects running on the wrong machine. +Consider a hypothetical compiler that can auto-detect that a build machine has many cores, and enables an optimization upon detection incompatible with machines with fewer cores. While Nix will generate different hashes if the platform architecture changes, say from X86 to ARM, it will not consider a machine with many cores different from one with fewer. So our example optimizing compiler could cause a frustrating problem. A local build on a machine with few cores may work as expected. But if a cache had a optimized build from a machine with many cores, it would be pulled down for the same hash, as a substitute for a local build. This optimization would lead to defects running on the wrong machine. Note that in general, we benefit from downloading and running packages built on more powerful machines, and in almost all cases, the clever optimizations of various compilers are portable. diff --git a/doc/nix-introduction.org b/doc/nix-introduction.org index 0c60124..ee562fd 100644 --- a/doc/nix-introduction.org +++ b/doc/nix-introduction.org @@ -1,6 +1,21 @@ #+title: Deciding to use Nix #+setupfile: internal/links.org +* Org-mode setup :noexport: + +This document is written in a project-agnostic way to be copied to other +projects that use Nix. + +** Variables + +We set variables in =internal/params.el= and access those settings with the +following macros. + +#+macro: get (eval (concat $2 (alist-get (intern $1) (car (read-from-string (f-read "internal/params.el")))) $3)) +#+macro: nix-latest {{{get(nix-latest)}}} +#+macro: nix-stable {{{get(nix-stable)}}} +#+macro: nixos-latest {{{get(nixos-latest)}}} + * About this document This document introduces the [[nix][Nix package manager]] and highlights some motivations @@ -171,7 +186,7 @@ builds more easily without worrying about deviations in environment configuration. Nix has a central substituter at https://cache.nixos.org, but there are -third-party ones as well, like [[cachix][Cachix]] and [[garnix][Garnix]]. Before building a package, the +third-party ones as well, like [[garnix][Garnix]] and [[cachix][Cachix]]. Before building a package, the hash for the package is calculated. If any configured substituter has a build for the hash, it's pulled down as a substitute. A certificate-based protocol is used to establish the trust of substituters. Between this protocol and the @@ -343,22 +358,42 @@ the following problems: - we have to accept some added training hurdles since the documentation of flakes is tucked behind documentation of non-flake usage. +*** Nix quick releases compete with stability + +The latest major version of the Nix package manager is currently Nix +{{{nix-latest}}}, but NixOS {{{nixos-latest}}}, the latest stable release of +NixOS, uses Nix {{{nix-stable}}}. NixOS is the primary way the Nix package +manager gets used in the field. Far fewer users install Nix as a package manager +atop another operating system. From a community perspective it makes sense to +consider Nix {{{nix-stable}}} the stable release of the package manager. This +version gets the most scrutiny and critical bug fixes. + +As mentioned above, there are strong reasons to use still-experimental features, +particularly flakes. However, APIs and calculated hashes change too frequently +in experimental features from version-to-version. By sticking with the version +used in NixOS, we get less breaking changes. For example, the [[file:../flake.lock][flake.lock]] file +included with this project has calculated hashes for dependencies. These hashes +were computed with Nix {{{nix-stable}}}, and could change with later versions. + +For these reasons, the [[file:nix-installation.org][installation guide included with this project]] recommends +installing Nix {{{nix-stable}}}, rather than the latest official release. + ** A few gaps in determinism Nix offers world-class build determinism, especially with flakes. But it's important to understand that this determinism is not infallible. To date, no build system can claim to provide flawless determinism. -Known gaps involve corner cases like the following. Consider a hypothetical -compiler that can auto-detect that a build machine has many cores, and enables -an optimization upon detection incompatible with machines with fewer cores. -While Nix will generate different hashes if the platform architecture changes, -say from X86 to ARM, it will not consider a machine with many cores different -from one with fewer. So our example optimizing compiler could cause a -frustrating problem. A local build on a machine with few cores may work as -expected. But if a cache had a optimized build from a machine with many cores, -it would be pulled down for the same hash, as a substitute for a local build. -This optimization would lead to defects running on the wrong machine. +Consider a hypothetical compiler that can auto-detect that a build machine has +many cores, and enables an optimization upon detection incompatible with +machines with fewer cores. While Nix will generate different hashes if the +platform architecture changes, say from X86 to ARM, it will not consider a +machine with many cores different from one with fewer. So our example optimizing +compiler could cause a frustrating problem. A local build on a machine with few +cores may work as expected. But if a cache had a optimized build from a machine +with many cores, it would be pulled down for the same hash, as a substitute for +a local build. This optimization would lead to defects running on the wrong +machine. Note that in general, we benefit from downloading and running packages built on more powerful machines, and in almost all cases, the clever optimizations of