Nix's build system should filterSource or similar

[Ericson2314]

Right now, changing flake.nix etc. in any way rebuild everything. These mass rebuilds make tweaking those files really annoying. It would be nice if we could filter those files out from the src of building Nix, so that this no longer happens.

[edolstra]

This would probably be addressed by #3121, so I'll close this one.

[Ericson2314]

@edolstra err it is not clear to me that it is? Whether or not data is coppied eagerly or lazily, there is still the question of what files are in-scope.

Also, it's not just a matter of removing flake.nix: other nix files like docker.nix should also be filtered out. Also the source should be split so the perl bindings don't cause the main stuff to rebuild and vice-versa. Finally, we do need to download the entire source for dependent flakes, but we should filter/chop up their sources just the same.

This makes me think the issue is more related to #1767? If we in general do need to fetch the whole thing, and then filter source after, I think the solution is probably making CA derivations that re-filter the source. The filtered source after all is needed for build-time only, at eval time it's fine to have extra stuff because eval cache mass rebuilds are not yet a thing people complain about.

(Maybe CA derivations that only produce a single output without references can be stabilized first, if need be, since they don't get into those thorny questions about closure coherence.)

[lilyball]

Also see #5551 regarding a potential source filter for path flakes.

[thufschmitt]

I agree with @Ericson2314 , this is orthogonal to #3121 and just tied to the fact that a number of files in the repo aren’t meaningful for the build, so changing them shouldn’t cause a rebuild.

(Note that this is an issue with having the .nix files embedded in the repo in general, but I don’t think there’s a proper general solution, except using builtins.path everywhere)

[Ericson2314]

@regnat Yeah, without pure eval, various stop-gap solutions work. So it is "worse" with flakes for reasons that are not exactly flake's fault.

[Ericson2314]

Oh, turns out just referencing sub-paths works fine with pure eval, so I just did that in #5589.

[stale]

I marked this as stale due to inactivity. → More info