cannot consume Tidal flake after switch to Link submodule

[ahihi]

it seems that #1119 makes the Tidal flake unbuildable when used as an input to another flake. for example, my nix-darwin flake includes

tidal.url = "github:tidalcycles/Tidal";

and when rebuilding my system, i now get the 'abl_link.h' file not found error.

i am not sure how to correctly pass submodules=1 in this situation. ive tried the following, which dont work:

• changing the input to tidal.url = "github:tidalcycles/Tidal?submodules=1";
• darwin-rebuild switch --flake .?submodules=1
• darwin-rebuild switch --flake .?submodules=1#

[sss-create]

you don‘t have to specify .?submodules=1 in the URL but as part of the nix build command

Tidal/.github/workflows/nix.yml

Line 47 in 08f0416

run: nix build '.?submodules=1#default' --print-build-logs

I an not sure how darwin-rebuild is different to the simple „nix build“ command. Can I test this somehow on Linux?

[ahihi]

the Linux equivalent would be nixos-rebuild, provided youre running NixOS of course (nix-darwin is basically "NixOS on top of macOS"). i do also have a NixOS install myself - i can test there later

[sss-create]

Looks like we could add a darwin-rebuild to our CI.

[sss-create]

@ahihi could you provide a minimum (not) working example for your flake in flake config?

[ahihi]

here: https://github.com/ahihi/tidal-flake-test

fails the same way on both nix-darwin and NixOS.

[ahihi]

btw, given the build complications that arise from submodules (not just this one - we just had someone on discord wondering why a clone & cabal build no longer works, and i expect there will be others), have you considered a git subtree as an alternative?

[sss-create]

Probably a good alternative to keep the burden off the user. What do you say, @yaxu?

Sorry for breaking it in the first place.

[sss-create]

A problem is that Link itself uses submodules again. It looks like subtrees + submodules aren't a friendly combination.

edit: So it seems to me, that a subtree is very similar to copying the Link code, as done before; with the difference that there is a dedicated git command to update the copied code.

[ahihi]

A problem is that Link itself uses submodules again. It looks like subtrees + submodules aren't a friendly combination.

oh! yeah, that makes it tricky.

edit: So it seems to me, that a subtree is very similar to copying the Link code, as done before; with the difference that there is a dedicated git command to update the copied code.

yes pretty much, and it also gives you the other project's history.

[sss-create]

Later on, when Tidal is released and installed the cabal --lib way, submodules do not play a role anymore, do they?
I think we can solve the Nix build issues, so people who want clone the repo will be able to do, with submodule awareness as a tradeoff. But maybe the tradeoff is too big?

[ahihi]

indeed, outside of Nix this is mainly a concern for people who want to run the dev branch for any reason.

generally, i think subtrees > submodules > manually vendoring the code. since Link using submodules itself makes the subtree approach infeasible, the submodule seems like the best available approach.

i also believe the Nix issue can be solved, im just very unclear on how this is supposed to work, in which context (nix build vs darwin/nixos-rebuild) and under which Nix versions (i saw something about a possible regression in Nix 2.24). will try to dig some more later

[yaxu]

I don't have any useful knowledge/experience/insights to share on this sorry! Hope the nix issue works out

[ahihi]

not sure if the problem is really related to darwin/nixos-rebuild after all. i updated https://github.com/ahihi/tidal-flake-test to also re-export tidal-ghci as the default package, and tried nix build '.?submodules=1#default'. this also fails with the missing abl_link.h.

[ahihi]

so the test can really be reduced to

{
  description = "flake to test consuming the Tidal flake";

  inputs = {
    tidal.url = "github:tidalcycles/Tidal";
  };

  outputs = { self, tidal }:
    tidal;
}

[ahihi]

ok, it looks like using tidal.url = "git+https://github.com/tidalcycles/Tidal?submodules=1"; instead of the github:… url works and lets me do both a nix build . (on the test flake) and a darwin-rebuild build --flake . (on my nix-darwin flake). no need to pass submodules=1 to these commands as it is specified in the Tidal flake url, which is great.

according to NixOS/nix#4423 (comment), this does introduce possible nondeterminism, but that is something to be addressed on the Nix side and i think i can live with it for now :)

[ahihi]

NixOS/nix#12421 was just merged, which should help here!