Nixpkgs upgrade #8

… is detected" We have a safer way of doing this. This reverts commit dc3ccf0.

As a prelude to making "or" work like a normal variable, emit a warning any time the "fn or" production is used in a context that will change how it is parsed when that production is refactored. In detail: in the future, OR_KW will be moved to expr_simple, and the cursed ExprCall production that is currently part of the expr_select nonterminal will be generated "normally" in expr_app instead. Any productions that accept an expr_select will be affected, except for the expr_app nonterminal itself (because, while expr_app has a production accepting a bare expr_select, its other production will continue to accept "fn or" expressions). So all we need to do is emit an appropriate warning when an expr_simple representing a cursed ExprCall is accepted in one of those productions without first going through expr_app. As the warning message describes, users can suppress the warning by wrapping their problematic "fn or" expressions in parentheses. For example, "f g or" can be made future-proof by rewriting it as "f (g or)"; similarly "[ x y or ]" can be rewritten as "[ x (y or) ]", etc. The parentheses preserve the current grouping behavior, as in the future "f g or" will be parsed as "(f g) or", just like "f g anything-else" is grouped. (Mechanically, this suppresses the warning because the problem ExprCalls go through the "expr_app : expr_select" production, which resets the cursed status on the ExprCall.)

This is better for privacy and to avoid leaking netrc credentials in a MITM attack, but also the assumption that we check the hash no longer holds in some cases (in particular for impure derivations). Partially reverts NixOS@5db358d.

Since NixOS#8766, invalid base64 is rendered in errors, but we don't actually want to show this in the case of an invalid private keys. Co-Authored-By: Eelco Dolstra <[email protected]>

…eak-private-key-on-error Ensure error messages don't leak private key

Most of the time people run single tests for debugging reason, so it's a sane default to have them see all the console output. This commit still retains the section about running tests directly with meson, because in some debugging cases it's just nice to have less abstractions i.e. when using strace.

builtin:fetchurl: Enable TLS verification

Fix typo

* docs: specify that flake.lock files are JSON Recently, I decided that I was going to write some code that would parse flake.lock files. I went to the Nix Reference Manual in order to look up information on the format of flake.lock files, and I realized that a key detail was missing from the Nix Reference Manual: it never says that flake.lock files are JSON files. This commit fixes that issue. This commit makes sure to specify that flake.lock files are encoded in UTF-8. Confusingly, there’s multiple different JSON standards. Neither ECMA-404, 2nd Edition [1] nor ISO/IEC 21778:2017 [2] mention UTF-8. RFC 8259 requires UTF-8, but only sometimes [3]. I chose to explicitly specify that flake.lock files are UTF-8 in order to avoid any possible ambiguities from the JSON standards. [1]: <https://ecma-international.org/publications-and-standards/standards/ecma-404> [2]: <https://www.iso.org/standard/71616.html> [3]: <https://www.rfc-editor.org/rfc/rfc8259.html#section-8.1>

When working on speeding up the CI, I triggered a race condition in the creation of the tarball cache. This code now instead will ensure that half-initialized repositories are no longer visible to any other nix process. This is the error message that I got before: error: opening Git repository '"/Users/runner/.cache/nix/tarball-cache"': could not find repository at '/Users/runner/.cache/nix/tarball-cache'

create git caches atomically

This method is marked as `noexcept`, but `enqueueFileTransfer()` can throw `Interrupted` if the user has hit Ctrl-C or if the `ThreadPool` that the thread is a part of is shutting down.

same fix as in 04a47e9

…che-reference Fix reference to HTTP Binary Cache Store in docs

…xception HttpBinaryCacheStore::getFile(): Fix uncaught exception

This patch has been manually adapted from lix-project/lix@14dc84e Tested with: $ NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }' Finished at 16:57:50 after 1s warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=' this derivation will be built: nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) google.com> error: nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) google.com> … writing file '/nix/store/0zynn4n8yx59bczy1mgh1lq2rnprvvrc-google.com' nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) google.com> nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) google.com> error: unable to download 'https://google.com': Problem with the SSL CA cert (path? access rights?) (77) error setting certificate file: /nix/store/nlgbippbbgn38hynjkp1ghiybcq1dqhx-nss-cacert-3.101.1/etc/ssl/certs/ca-bundle.crt nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) error: builder for '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' failed with exit code 1 Now returns: nix-env % NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }' Finished at 17:05:48 after 0s warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=' this derivation will be built: nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory) error: hash mismatch in fixed-output derivation '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv': specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=

Signed-off-by: Jordan Justen <[email protected]>

This leads to confusion about what the command does. E.g. NixOS#9359 - Move the description up - Remove details about the individual formatters

doc/nix fmt: Do not document behavior we do not control

nix-ng: add meta.mainProgram

tests/functional/flakes/run: fix tests in macOS devshell

Co-authored-by: Jörg Thalheim <[email protected]>

A test added recently checks that when trying to deserialize a NAR with two files that Unicode-normalize to the same result either succeeds on Linux, or fails with an "already exists" error on Darwin. However, failing with an "already exists" error can in fact also happen on Linux, when using ZFS with the proper utf8 and Unicode normalization options set. This commit fixes the issue by not assuming the behavior from the current system, but just by blindly checking that either one of the two aforementioned possibilities happen, whether on Darwin or on Linux. Additionally, we check that the Unicode normalization behaviour of nix-store is the same as the host file system.

…formed-uri-query fix: warn on malformed URI query parameter

The previous documentation was inaccurate, stating that it would not update existing inputs. However these inputs will be updated if they are outdated (for example the version of an existing input has been changed). The new text properly reflects this behaviour.

Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from V28 to 29. This release includes the previously tagged commit. - [Release notes](https://github.com/cachix/install-nix-action/releases) - [Commits](cachix/install-nix-action@V28...v29) --- updated-dependencies: - dependency-name: cachix/install-nix-action dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]>

Improve documentation of `nix flake lock`

eval: remove superfluous strdup

fix: include missing header when GC is disabled

…achix/install-nix-action-v29 Bump cachix/install-nix-action from V28 to 29

Split ignoreException for destructors vs interrupt-safe

…ruction doc: don't use bash substitution in install command

…d-or libexpr: deprecate the bogus "or"-as-variable

doc/manual: Fix some broken fragments

These versions are more than 3 years old and were very early in the existence of CA derivations support (which was and is experimental), so they're unlikely to still exist in the real world. So let's get rid of support for them.

…ma-migrations Cleanup: Remove migrations for ancient CA schemas

This caused nlohmann/json.hpp to leak into a lot of compilation units, which is slow (when not using precompiled headers). Cuts build time from 46m24s to 42m5s (real time with -j24: 2m42s to 2m24s).

experimental-features.hh: Don't include json-utils.hh

Fix typo in uninstall.md

…stream

fix passing CA files into builtins:fetchurl sandbox

Co-authored-by: Jörg Thalheim <[email protected]>

Fix NAR tests on Linux+ZFS+normalize

Progress on NixOS#9205

Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 29 to 30. - [Release notes](https://github.com/cachix/install-nix-action/releases) - [Commits](cachix/install-nix-action@v29...v30) --- updated-dependencies: - dependency-name: cachix/install-nix-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>

…achix/install-nix-action-30 build(deps): bump cachix/install-nix-action from 29 to 30

as requested Co-Authored-By: Robert Hensing <[email protected]>

This is unrelated to this PR, but requested in NixOS#11224 (comment) Co-Authored-By: Robert Hensing <[email protected]>

Co-Authored-By: Robert Hensing <[email protected]>

Identified in NixOS#11224 (comment) Co-Authored-By: Robert Hensing <[email protected]>

Co-Authored-By: Robert Hensing <[email protected]

Misc changes

Co-Authored-By: Qyriad <[email protected]> Co-Authored-By: Robert Hensing <[email protected]> Co-Authored-By: eldritch horrors <[email protected]> Co-authored-by: Jörg Thalheim <[email protected]> Co-authored-by: Tom Bereknyei <[email protected]>

This should help with issues like DeterminateSystems/nix-installer#1227, which currently just print "unable to start build process".

Build manual with Meson

…s-cleanup Slightly more `std::filesystem` for `nix eval`

…etup-error Propagate errors from early sandbox initialization to the parent

Fixes deprecation warning from nix build: warning: 'nix store ping' is a deprecated alias for 'nix store info'

Fix deprecated example from distributed-builds.md

Also, add an activity to show that we're downloading an s3:// file. Fixes NixOS#11674.

Remove superfluous `std::string` copy operations

If you have the Nix store mounted from a nonlocal filesystem whose exporter is not running as root, making the directory mode 000 makes it inaccessible to that remote unprivileged user and therefore breaks the build. (Specifically, I am running into this with a virtiofs mount using Apple Virtualization.framework as a non-root user, but I expect the same thing would happen with virtiofs in qemu on Linux as a non-root user or with various userspace network file servers.) Make the directory mode 500 (dr-x------) to make the sandbox work in this use case, which explicitly conveys our intention to read and search the directory. The code only works because root can already bypass directory checks, so this does not actually grant more permissions to the directory owner / does not make the sandbox less secure.

doc: note that `nix eval` is eager --------- Co-authored-by: Robert Hensing <[email protected]>

git-utils: fix x86_64-w64-mingw32 build

... and remove a few unused arguments. This adds pkg-config to a two or three packages that don't use it, but we shouldn't let that bother us. It's like our personal stdenv.

and: - move pkg-config out of mkMesonDerivation, for components that don't produce any executable code

I.e. when not all entries underneath a directory X follow eachother, but there is some entry Y that isn't a child of X in between. Fixes NixOS#11656.

builtins.fetchurl: Fix segfault on s3:// URLs

libstore: Make our sandbox pivot_root directory accessible to ourself

…-tarballs Handle tarballs where directory entries are not contiguous

Fixes NixOS#11660.

This is needed to avoid this mesonbuild/meson#13774 when we go back to making our subproject directory `src`.

We got rid of this in c7ec336 because of bug mesonbuild/meson#13774, but in the previous commit we renamed the manual source directory, which avoids it. Now we can change it back.

Clean up the `package.nix` files

Make the subproject dir `src` again

Otherwise, if checkInterrupt() in any of the supported store operations would catch onto a user interrupt, the exception would bubble to the thread start and be handled by std::terminate(): a crash.

Introduced in 8f6b347 without explanation. Throwing anything that's not that is a programming mistake that we don't want to ignore silently. A crash would be ok, because that means we/they can fix the offending throw.

…odified path fetcher: Allow the lastModified attribute to be overriden again

Co-authored-by: Cole Helbling <[email protected]>

…ut-of-thread Fix `Interrupted` falling out of thread crash

Let's have one canonical location for the team membership.

maintainers/README.md: Remove the list of team members

maintainers/onboarding: Start documenting

Everything that is a separate subproject should live in the subprojects directory. Progress on NixOS#2503 This reverts commit 451f8a8.

Move unit tests to the location Meson expects them to be

If the call is inside the assertion, then in non-assert builds the call would be stripped out. This is highly unexpected.

…y-out-of-assert fix(nix/eval.cc): move call to `fs::create_directory` out of `assert`

doc: Explain why tryEval does not return the message

docs: clarify syntax for escaping dollar curlies

The buildoutput of doxygen often hides important build warnings and errors behind a wall of text.

This overall seems like insecure tmp file handling to me. Because other users could replace files in /tmp with a symlink and make the nix-shell override other files. fixes NixOS#11470

make doxygen build more silent

fix env-vars beeing written to `/tmp`

tests/functional/help.sh calls nix-* commands with option --help if nix is built without documentation the option --help throws an error because the man page it wants to display is missing

nix shell/run: Use overlayfs

Co-Authored-By: Théophane Hufschmitt <[email protected]>

This works because the `builder` and `args` variables are only used in the non-builtin code path. Co-Authored-By: Théophane Hufschmitt <[email protected]>

std::stringbuf is defined in <sstream>

Fix meson build on darwin

the default int64_t max was still overflowing for me, when this was dumped as json (noticed during building the manual). So making 0, the default and define it as "no warnings" fixes the situtation. Also it's much more human-readable in documentation.

Add a test for chroot stores

warn-large-path-threshold: define 0 as number to disable warnings

nix/tests: run test help.sh only if nix is built with documentation

OpenBSD dynamic libraries never link to libc directly. Instead, they have undefined symbols for all libc functions they use that ld.so resolves to the libc referred to in the main executable. Thus, disallowing undefined symbols will always fail

Nix fails to build on OpenBSD with a linking error due to a non-found symbol in boehm-gc. Just disable the GC until we can find a proper workaround.

OpenBSD doesn't support `lutimes`, but does support `utimensat` which subsumes it. In fact, all the BSDs, Linux, and newer macOS all support it. So lets make this our first choice for the implementation. In addition, let's get rid of the `lutimes` `ENOSYS` special case. The Linux manpage says > ENOSYS > > The kernel does not support this call; Linux 2.6.22 or later is > required. which I think is the origin of this check, but that's a very old version of Linux at this point. The code can be simplified a lot of we drop support for it here (as we've done elsewhere, anyways). Co-Authored-By: John Ericson <[email protected]>

Fix building on OpenBSD

Add support for `utimensat` as an alternative to `lutimes`

Refactor `import`

docs: update distributed-builds.md

Ref NixOS#10795

Correct a few typos. Make explicit that FSO acronym refers to File System Object.

Expand shellcheck coverage in functional tests

Co-Authored-By: Robert Hensing <[email protected] Co-authored-by: Dan Baker <[email protected]>

…ty-release maintainers: add checklist for security releases

fix: make sure directory exists before using `ln`

* doc/rl-2.19: add entry for always-allow-substitutes option Fixes NixOS#9427.

* doc/manual: Add 'Debugging Nix' section This commit adds a new 'Debugging Nix' section to the Nix manual. It provides instructions on how to build Nix with debug symbols and how to debug the Nix binary using debuggers like `lldb`. Co-authored-by: Jörg Thalheim <[email protected]> Co-authored-by: Valentin Gagarin <[email protected]>

For example, we can't even clone the repository on Windows! error: invalid path 'tests/functional/nul.nar' fatal: unable to checkout working tree

operators are an everyday thing in the Nix language, and this page will hopefully be consulted by many users. string contexts are quite exotic, and not linking to the detailed explanation will require readers to figure out manually what this is about, or worse, skim over and run into problems later.

…context docs: add links to string context documentation

This should make `_NIX_TEST_ACCEPT=1` work again, fixing NixOS#11369. Progress on NixOS#2503

…l-tests No longer copy functional tests to the build dir

…ld-tmpdir-env-fix Fix compilation of nix-build on Windows

…l-character Rename nul.nar because nul is a special name in Windows

Anything less won't compile because we're using GetCurrentThreadStackLimits from Windows 8.

Fix compilation of eval under Windows

…ironment-set-env Add flag `set-env-var` to `MixEnvironment`, and rename old ones

Co-authored-by: Robert Hensing <[email protected]>

…unctional-tests Meson for nixos functional tests

Co-authored-by: Robert Hensing <[email protected]>

…-sdk Set minimum Windows API to Windows 8

…g-meson Test against tests using Meson-built Nix

Getting ready for the complete switch-over Progress on NixOS#2503

…s-meson `hydraJobs.build{NoGc,ReadlineNoMarkdown}` build using Meson

It had gotten rather big. Hopefully we'll eventually have some generic infra for a "multi-package dev shell" and not need so much code for this, but until then it's better in a separate file.

We have per-stdenv package sets, so we should be using them.

When we get rid of the make build system, we would be missing things. Incuding these packages' deps ensure we don't miss things.

…indows-version Set Windows API version in Meson

…ixes Meson fixes for msys2 builds

Unfortunately `StringSource` class is very easy was very easy to misuse because the ctor took a plain `std::string_view` which has a bad habit of being implicitly convertible from an rvalue `std::string`. This lead to unintentional use-after-free bugs. This patch makes `StringSource` much harder to misuse by disabling the ctor from a `std::string &&` (but `const std::string &` is ok). Fix affected tests from libstore-tests. Reformat those tests with clangd's range formatting since the diff is tiny and it seems appropriate.

`lib.concatMapAttrs` instead of `lib.mapAttrs'` and `lib.nameValuePair` Co-authored-by: Robert Hensing <[email protected]>

Dev shell improvements

…ibstore-tests fix(libstore-tests): remove use-after-free bug for `StringSource`

Cross-compilation of curl is broken in nixpkgs. Therefore we disable it until nixpkgs fixes the underlying issues.

This reverts commit e44d98b.

Nixpkgs upgrade #8

Are you sure you want to change the base?

Nixpkgs upgrade #8

Commits on Sep 20, 2024

Commits on Sep 23, 2024

Commits on Sep 24, 2024

Commits on Sep 25, 2024

Commits on Sep 26, 2024

Commits on Sep 27, 2024

Commits on Sep 28, 2024

Commits on Sep 29, 2024

Commits on Sep 30, 2024

Commits on Oct 1, 2024

Commits on Oct 2, 2024

Commits on Oct 3, 2024

Commits on Oct 4, 2024

Commits on Oct 5, 2024

Commits on Oct 6, 2024

Commits on Oct 7, 2024

Commits on Oct 8, 2024

Commits on Oct 9, 2024

Commits on Oct 10, 2024

Commits on Oct 11, 2024

Commits on Oct 12, 2024

Commits on Oct 13, 2024

Commits on Oct 14, 2024

Commits on Oct 16, 2024

Commits on Oct 17, 2024

Commits on Oct 18, 2024

Commits on Oct 20, 2024

Commits on Oct 21, 2024

Commits on Oct 22, 2024

Commits on Oct 23, 2024

Commits on Oct 24, 2024

Commits on Oct 26, 2024

Commits on Oct 27, 2024

Commits on Oct 30, 2024

Commits on Oct 31, 2024

Commits on Nov 1, 2024

Commits on Nov 3, 2024

Commits on Nov 4, 2024

Commits on Nov 5, 2024

Commits on Nov 6, 2024