Low level <drvPath>^<outputName> installable syntax to match existing <highLevelInstallable>^<outputNames> syntax

[Ericson2314]

#6426 added ^<output-names> to high level installables (flakes ones and attribute paths). This adds it to our low level installable --- a plain store path --- to match.

I think this is important because it greatly helps "complete" the expressive power of Nix's low-level "plumbing" commands, and doing that (and eventually stabilizing them) is a good cost / benefit win.

In addition:

One ingredient for #7261, but should be good even if we reject that issue.
Also important for RFC 92 (tracking issue #6316).

[matthewbauer]

I think CmdProfileInstall also needs to change to reflect this:

https://github.com/NixOS/nix/blob/master/src/nix/profile.cc#L267-L270

so only outputs listed in BuildableFromDrv "outputs" are installed.

[thufschmitt]

I'm not sure what this brings implementation-wise compared to #4494 (except obviously for the doc and test which are obviously missing in the latter). In particular it seems to me that having an extra Installable type just for that is a lot of boilerplate for not much benefit. Or do I miss something?

[Ericson2314]

@regnat I'm afraid I didn't even realize you had a PR this did this!

[Ericson2314]

@regnat I think the simpler and pure (no DB access) toBuildables is notable. (To be clear, yours made the problem no worse, just inherited the complexity) I am very skeptical of all the complexity we have today (along with --derivation) and so hope this can be the beginning of getting rid of it.

[dpulls]

🎉 All dependencies have been resolved !

[fogti]

This doesn't solve the shell 'escape needed' problems. I'm unsure if ! is a wise choice. It shows unwanted behavoir e.g. in bash:

$ nix profile install nixpkgs!lol
bash: !lol: event not found

(e.g. similiar to #4686, but with ! instead of #)

[Ericson2314]

Yeah we didn't settle on a surface syntax yet.

[fogti]

Currently, this syntax is (already) used with CA-drvs, which leads to problems with nix-collect-garbage, because it terminates when it finds any store which uses that syntax. Maybe this should be fixed in a smaller PR earlier if possible, because this one seems to take another while until merged (and I'm not sure if this PR does really fix that annoyance).

[thufschmitt]

Currently, this syntax is (already) used with CA-drvs, which leads to problems with nix-collect-garbage, because it terminates when it finds any store which uses that syntax

Unless I missed something, these have only been created because of a now-closed bug (#5176), and it only concerns lockfiles. Is there more of these?

[Ericson2314]

Yeah this has nothing to do with that one, which @regnat fixed. This is just making an internal syntax exposed to the user.

In this case, we in fact rely on ! being disallowed in store paths, so there is no ambiguity.

[Ericson2314]

I should also mention that nix-store -r /nix/store/3v8v9alqxfhif646mrvrafv2hmw60ypx-cabal2nix-reflex.drv!out is already valid syntax, hah.

[tomberek]

To clarify, this is existing syntax that happens to not function with installables, but would otherwise be allowed. Would this leak into things like profiles or machine-readable output?

[Ericson2314]

This is preexing syntax for some legacy commands (e.g. nix-build) and used internally (though I modernized it). Basically, see DerivedPath and StorePathWithOutputs in the code base, I have made the former replace the latter where possible.

This specific change won't leak anywhere, this is just changing the CLI syntax for installables. In way, the leaking behind the scenes is already done! --- thanks to those DerivedPath refactors underneath the hood.

This, or something like this, is part of RFC 92, so we should figure out what the experimental feature is called and get this merged. I had been waiting to hopefully add some more tests firsts for the status quo first, but if that is controversial we should just go ahead with this and the other parts of RFC 92.

[fricklerhandwerk]

Discussed in Nix team meeting 2022-11-28:

• @fricklerhandwerk: what are installables anyway?
  • @Ericson2314: when you do nix build foo bar baz, each of the arguments could be
    • store derivation path
    • store path
      • which could be not built but substitutable
    • path to Nix expression
    • flake reference
  • @Ericson2314: we can already do nix build nixpkgs#foo^man (i.e., flake references), but not for store paths such as nix build /nix/store/<hash>-foo^man.
  • Are there any objections that we want to have this or the implementation strategy?
    • agreement on both

-> to review

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2022-11-28-nix-team-meeting-minutes-12/23730/1

[Ericson2314]

Note it previously read the derivation and returned all its output names explicitly here, but that didn't work e.g. when the drv file itself needs to be downloaded and so we cannot yet read it.

[Ericson2314]

OK @edolstra, I think all review from today is now addressed!

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2022-12-12-nix-team-meeting-minutes-16/24119/1

[ghost]

Hey I know this is the bikesheddiest of bikesheds, but I just realized we could have used / instead of ^:

nix build /nix/store/09wshq4g5mc2xjx24wmxlw018ly5mxgl-bash-5.2-p15.drv/man

... since Nix strictly forbids a top-level store path which isn't a derivation from having the suffix .drv. Since derivations are files, not directories, the string above can never be a valid filesystem path.

Who cares, you ask? Maybe it's just me, but the 6 key on a QWERTY keyboard is farther from the home row position than any other key, meaning that 6 and ^ are mistyped more often than any other character.

Like I said, bikesheds. Glad I got that out of my system. We now return you to your regularly-scheduled software development...

[Ericson2314]

@amjoseph-nixpkgs I think some command allow you specify a store path with trailing path relative to it? It could get a git confusing in that case. On a conceptual level I want /nix/store/09wshq4g5mc2xjx24wmxlw018ly5mxgl-bash-5.2-p15.drv^man/share/man to work too, but I realize that can also be confusing and maybe it's better to stick to some sort of ${...} notation.