Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RFC 0123] Flake names #123

Draft
wants to merge 15 commits into
base: master
Choose a base branch
from
Draft

[RFC 0123] Flake names #123

Changes from 2 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
9515060
📂 Add RFC “flake-names”
schuelermine Mar 12, 2022
08d1fc5
[RFC 0123] Flake names
schuelermine Mar 12, 2022
bae96ea
Remove statement about flake registry interaction with flake names
schuelermine Mar 13, 2022
53b1dc7
Add example to flake names
schuelermine Mar 13, 2022
a108a3b
Fix invalid URL in flake names example
schuelermine Mar 13, 2022
9acd46f
Add note that nix flake show should use this, too
schuelermine Mar 13, 2022
23e44a1
Update 0123-flake-names.md
schuelermine Mar 13, 2022
bb09d85
(Flake names) Move & demote interactions
schuelermine Mar 13, 2022
b077d31
(flake names)Change short rev to tag <
schuelermine Mar 13, 2022
79d2e61
Remove references to discoverability and separate statement about git…
schuelermine Mar 14, 2022
0cb078f
Update rfcs/0123-flake-names.md
schuelermine May 20, 2022
e3b651f
Remove source control mention
schuelermine May 20, 2022
7811f0e
Merge branch 'master' of https://github.com/schuelermine/nix-rfcs
schuelermine May 20, 2022
d8c2e19
Slightly rewrite RFC 0123
schuelermine May 21, 2022
c3bdc8d
RFC 0123: Slightly change wording
schuelermine Jun 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions rfcs/0123-flake-names.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
feature: flake-names
start-date: 2022-03-12
author: Anselm Schüler
co-authors: None
shepherd-team: None
shepherd-leader: None
related-issues: None
---

# Summary
[summary]: #summary

Flakes can declare the field `name`.
It represents the name of the flake.
The derivations for a flake are no longer called `source`, but use the flake name.
schuelermine marked this conversation as resolved.
Show resolved Hide resolved

# Motivation
[motivation]: #motivation

Flake-centric workflows often end up with a lot of derivations named “source”, and it’s difficult to navigate this.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s probably worth a bit more explanations: for most intents and purposes, the store is supposed to be an opaque thing, and not meant to be directly “navigated”.

Now it is a very leaky abstraction, so my remark probably doesn’t really hold in practice, but it would be good to make this explicit (maybe show some concrete examples of where having all the sources be called “source” is an issue)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an issue if you want to know what a nix command is doing at the moment. It often just displays “building XXXX-source”

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be wrong, but that exact message shouldn’t happen (not because of flakes at least). This message happens when Nix builds a derivation, but flake sources aren’t derivations, they are directly instantiated during the evaluation (with builtins.fetchTree) using a different code path with amongst other things a more informative display.

What you might get though is something like “fetching source from https://cache.nixos.org” (because although they aren’t derivations, they can be substituted from the binary cache). Probably we could improve the message in that case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify: That message can happen for reasons unrelated to flakes, because a number of fetchers from nixpkgs hardcode or default name to “source” (and these are regular derivations, so you can get that message)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that’s good to know, sorry for not knowing that.

Copy link
Author

@schuelermine schuelermine May 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About the store path being meant to be opaque: In my opinion that’s just practically not true at all, and the fact that derivation names exist at all proves it. If anything, at least this is useful for people that want to spelunk in the depths of what Nix does.

I also regularly use store paths to figure out what package or other source provides some binary in my profile. I have a shell function goto-bin for going into the directory of the real path of an executable, and I sometimes skim realpath output to see a derivation name.

Also, the discoverability and usability of flakes needs to be improved. Current commands mostly show technical information. This would be a step in the right direction.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Unique names are useful for the purpose of finding duplicate things caused by diamond dependencies. (E.g. `a -> b -> d1`, `a -> c -> d2`. Without a unique name (`d`), it is not possible to correlate `d1` and `d2` reliably.)

# Detailed design
[design]: #detailed-design

A new supported property for flakes is introduced, `name`.
Running `nix flake metadata` on a flake that declares this field displays it at the top.
The derivation that contains the flake’s content is called `flake-source-${name}` or, if a short revision identifier is available, `flake-source-${name}-${shortRev}`.

# Examples and Interactions
[examples-and-interactions]: #examples-and-interactions

A flake that is in the registry should have its name match the registry identifier.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't that prevent the use of alternate names, or having multiple versions of the same flake in the registry?
(I use both)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s a good point

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it’d be a good rule for the global registry, though, but it’d probably be followed anyways, and would only hinder edge-cases


# Drawbacks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The big drawback here (where “big” would probably need to be refined, I’ve no idea how much of an issue it is in practice) is that since the name is part of the store path, two flakes with the same content but a different name will end-up in a different place in the store − which itself means that everything that depends on them would have to be rebuild.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But flakes with a different name will necessarily have different content? Or are the flake.* files not copied to the store?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point (I had a convoluted scenario in mind, but that’s probably way out of the scope of this RFC now that I think of it).

That’s actually highlighting an interesting design challenge: There’s currently a neat separation between the fetching code and the code that knows about what the flake is. inputs.foo = something gets translated to a call to builtins.fetchTree something, and its result is then fed to the outputs function. And the fetchTree part doesn’t know (nor has to know) that it’s fetching a flake.
But if the name gets defined in the flake (and used in the output path), then the fetchTree logic must be a bit more convoluted:

  • It has to know whether it is fetching a flake
  • When fetching a flake, it must get its flake.nix, evaluate it to find its name, and only then put it in the store with the correct name.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn’t it be better to have this conversation outside the review widget? I feel like this aspect of the discussion is somewhat hidden & non-chronological.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • When fetching a flake, it must get its flake.nix, evaluate it to find its name, and only then put it in the store with the correct name.

This can be solved by adding name to the lock file.
We can use the same trick for other fetch-related parameters, like whether to fetch submodules or not, whether to git-crypt decrypt, etc, all of which would also be nice to configure in flake.nix.
fetchTree remains unaware of flake-ness.

[drawbacks]: #drawbacks

This may cause clutter and additional maintenance.

# Alternatives
[alternatives]: #alternatives

Flake names could be handled entirely through outside means, with things like the global registry merely pointing to flakes under names.

# Unresolved questions
[unresolved]: #unresolved-questions

The name scheme could be changed. `flake-source-${name}-${shortRev}` could be too long.

# Future work
[future]: #future-work

Flake discoverability and usability needs to be improved.