[RFC 0139] Declarative filesystem layouts

[l0b0]

WIP, feel free to get involved.

@Lassulus @Mic92 As the biggest contributors to Disko, your ideas & contributions would be very welcome. I feel like declarative filesystem layouts could massively simplify one of the biggest, riskiest, and most long-standing complexities of installing any Linux distro.

[Lassulus]

I will try to chime in when I find some time, but if you want to do a jitsi session or something like that I could talk a bit about plans to integrate disko into nixpkgs. maybe one can sketch out the RFC after that :)

[AndersonTorres]

The idea of this is to use the Nix language to describe the hard disk layout?

[l0b0]

The idea of this is to use the Nix language to describe the hard disk layout?

Yes, ideally in such a way as to make sure the info can be used to both apply the structure (using any of the standard tools, like fdisk) and to declare them at runtime in for example /etc/fstab.

[samueldr]

One thing to consider in your write-up: what happens when the end-user changes the described layout on a running system?

In the past, non-commital lightweight discussions about that topic always ended-up at that dead-end, where declaratively describing this inherently stateful thing can't be handled through the lifecycle of NixOS systems, except at the very start before the first generation is booted, otherwise known as "installation".

Again, something to consider, please continue :).

[Lassulus]

Well we can't really change the filesystem of the system we are running on without losing a bunch of files. But maybe we can add a flag (similiar to how we handle the bootloader) to try to force the application of the filesystem changes. We could then kexec into a live system. rescue some files we care about, reformat the disks and then bootup into the new system. with a lot of files lost though

[maljub01]

A recent MacOS update forced filesystem changes on all devices (including removing /nix). This was done by adding a "Relocated Items" folder on the user's desktop and placing all the files there that would otherwise be deleted/lost. I remember thinking that this was a pretty neat solution. A similar idea is the lost+found directories used in some Linux filesystems. Maybe we can do something similar?

Maybe this could work:

1. Copy the files that would be deleted/lost to a path in the Nix Store.
2. Have a /nix/lost+found directory.
3. Add a garbage collection root to the Nix Store path from step 1 above under /nix/lost+found.

EDIT: Actually, it's probably a bad idea to use the lost+found name in case /nix is using its own filesystem that adds that directory itself. But there are plenty of other names that can be used.

[claes]

Could Ignition serve as an example?
https://coreos.github.io/ignition/examples/

[arianvp]

Note that we recently merged repart support which allows you to declaratively create your partitions and filesystems. Adding support for it to nixes-install is definitely something I'd like

NixOS/nixpkgs#232533

[l0b0]

Note that we recently merged repart support which allows you to declaratively create your partitions and filesystems. Adding support for it to nixes-install is definitely something I'd like

Excellent!

@nikstur Thank you! Especially like the test. Is this going to be in 23.05? Is there a PR with documentation on the way? And is it possible to use this to bootstrap a NixOS install?

[nyabinary]

Any status on this?

[l0b0]

@nyabinary There's been a lot of excellent feedback, and it sounds like people would be interested in the feature, but I don't feel qualified to lead this to completion by myself. As far as I'm concerned, there are a few possible next steps. Off the top of my head and in no particular order:

• Evaluate systemd-repart as a partial or complete solution to this. I'm not familiar with it, I haven't seen any examples of it, and I don't know whether it's appropriate for this use case.
• Evaluate the Ignition syntax as an alternative to Disko. I'm not familiar with either, but it looks like they could both do the job. On the other hand, we need to be wary of analysis paralysis and bikeshedding.
• Come up with common use cases (like initial setup when booting from USB/network, how to change an existing system, etc) and solutions. For example, maybe nixos-rebuild needs a new flag or subcommand to apply the configuration, or to check whether the configuration needs applying.
• Have a big get-together to work out what people are most interested in/worried by.
• Work out what would be a minimal first step towards this and implement that rather than designing a bunch up front.

[nyabinary]

Any status on this?

I feel like this plus impermanence would be great, but impermanence would def be out of scope for this RFC.

[nyabinary]

I can help/be the new author of this for now if nobody else steps up, I still think this is a great idea.

[Mic92]

Disko in the meantime also added support for updating some aspects of the disk configuration in an existing system. Systemd-repart supports some stuff as well but is still more limited.

[GetPsyched]

I can help/be the new author of this for now if nobody else steps up, I still think this is a great idea.

If you're still up for it and have the bandwidth to progress the RFC, please feel free to undraft, open for nominations and get things moving. I'm sure a lot of people, like myself, would like to see this materialise.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-09-02/51514/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-09-16/52224/1

[Mic92]

What is the motivation for this RFC? Don't we already have implementation? What are we trying to decide on?

[Geezus42]

I'm aware of Disko, but I'm not aware of any official way to declare filesystems.

[Frontear]

I'm aware of Disko, but I'm not aware of any official way to declare filesystems.

fileSystems.<name>? Could you elaborate on what exactly you mean by that? Or do you mean filesystems for a specific partition?

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-09-30/53690/1

[kevincox]

RFCSC:

This RFC has not acquired enough shepherds. This typically shows lack of interest from the community. In order to progress a full shepherd team is required. Consider trying to raise interest by posting in Discourse, talking in Matrix or reaching out to people that you know.

If not enough shepherds can be found in the next month we will close this RFC until we can find enough interested participants. The PR can be reopened at any time if more shepherd nominations are made.

See more info on the Nix RFC process here

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-10-28/55095/1

[Frontear]

I'm against this RFC, and projects like disko by proxy (no disrespect to Mic92 or any other contributors) because I feel it tries to bring declarative to something that just doesnt make sense in a declarative landscape.

Storage devices are an out of control property of systems, where one system may have an SSD (as nvme0n) and another have a HDD (as sda), and another may have both or multiple or anything! The problem is attempting to declare something about an unknown ahead-of-time. It's simply not realistic, and although projects like disko have made it easy enough to change one or two knobs and configure a new drive immediately, it doesnt fully change that nix isnt the best suited tool for such a thing. Frankly, nothing is, and nothing really can be unless guarantees are setup outside of the software context that promise a specific set of hardware availability, something which isnt guaranteed nor expressible at a level like nixpkgs, as its an individual requirement.

I think what we have now, with encouraging the use of labels and/or partlabels, is honestly the best approach for both a mix of declarative configuration whilst conforming to an outside scope. Anything beyond isnt feasible to support in nixpkgs specifically for the aforementioned reasons.

Apologies if my comment comes off harsh or disrespectful, I mean no disrespect. I just believe that such a function isn't bringing enough benefits to be worth supporting.

[RuRo]

Storage devices are an out of control property of systems, where one system may have an SSD (as nvme0n) and another have a HDD (as sda), and another may have both or multiple or anything! The problem is attempting to declare something about an unknown ahead-of-time.

Doesn't this argument apply equally to any other hardware-dependent parts of the configuration? For example, NVIDIA graphics, wired and/or wireless network, bluetooth, EFI stuff, virtualization, battery, temperature and fan management, anything that needs extra kernel modules, etc.

Why is defining your disk partitioning scheme any different from configuring power management for your laptop? Is it because the partitioning scheme persists between reboots? Well, it's not the only thing that persist, either. A lot of state SHOULD persist between reboots on a normal computer (even if you use something like impermanence, which isn't even the default).

So while I am not 100% sure if this needs to be a whole RFC, I definitely wouldn't say that this idea is infeasible, unreasonable or isn't worth supporting. 🤷

[Frontear]

Doesn't this argument apply equally to any other hardware-dependent parts of the configuration? For example, NVIDIA graphics, wired and/or wireless network, bluetooth, EFI stuff, virtualization, battery, temperature and fan management, anything that needs extra kernel modules, etc.

Valid point! As you said, that argument can definitely apply to these too, maybe I can make it more directly clear and say that it's not something I think needs first-class support, like hardware does in Nixpkgs. Granted, they all configure an "unknown", that unknown referring to physical hardware, but while things like GPU drivers, WiFi drivers and others are persistently important, disk layout and partitioning is usually a one and done thing. It doesn't need the first-class support because it's not really a problem after installation. Furthermore, trying to support it would either have to (a) reinvent whatever disko is doing, or just bring it upstream into nixpkgs, or (b) provide a very sparse evaluation that can generate a script to perform disk layout changes.

There's also the issue of when and how nixpkgs should even be doing this. It makes sense at nixos-install time, but beyond that, how would configuration of a system make use of a disk layout after installation time? It doesn't really serve a function within your config, and since the configuration largely pertains to the moving parts of an existing, functioning installation, disk layout management feels a bit amiss here.

So while I am not 100% sure if this needs to be a whole RFC, I definitely wouldn't say that this idea is infeasible, unreasonable or isn't worth supporting. 🤷

Valid, I should probably clarify I don't think its unreasonable as an idea itself, but I do think it's not a logical fit for nixos modules to contain such a script. Where and when would it even activate? Logically only once, at installation time, so then what happens after, does the activation script just ignore it forever after that, until a new situation arises? There's also the problem of the activation script being something that runs on a running system, not one that's being installed onto, so it wouldn't even activate anyways. The only reasonable thing that could be done is an external evaluation of the disk layout, and then probably a script generated off of that to create the partitions. Disko essentially already takes care of the implementation in their own way, and they do so by activating externally through their own evaluation. That's pretty much the only place where such an activation can happen anyways, and if nixpkgs were to offer first-class support, they'd have to do the same thing, which is honestly not ideal from a maintenance standpoint. It opens the can of worms that any/all one-off software can and/or should be merged upstream, and creates a maintenance burden on dealing with them. This is especially true because disk formatting is a sensitive procedure, and one wrong assumption upstream easily results in the loss of data, something which is a little too scary to provide first-class support for. At least with disko, its a focused project and contributors have a lot more time to be able to perform thorough testing and address complaints because it's a singular objective. Nixpkgs however is not able to provide such, and I don't think it should be expected to either.

[GetPsyched]

A polite reminder to start discussions as a review comment on the RFC file instead of discussing directly -- it helps keep the chat organised.

[Frontear]

If this does get accepted I think it should happen after a revamp to the Calamares installer, which from what I've seen (and heard as anecdotal evidence) is quite lacking and prone to not installing properly.

[Frontear]

This actually would be nice, independently of the RFC if it doesn't happen. Would make setting up new hosts a breeze, and would be especially nice if it could somehow grab mounts opts too

[chuangzhu]

We currently have config.sdImage for building images for non-UEFI platforms like Raspberry Pis.
That isn't very flexible, can only create an EXT4 rootfs and a FAT firmware partition. Anything other than that requires overriding system.build.sdImage.buildCommand, which is a bunch of commands.