consider a torrent for build inputs

[knupfer]

In a lot of cases, upstream sources or releases become unavailable (e.g. grsecurity) or even get modified (e.g. texlive) which breaks the package in older versions of nixpkgs.

It would be nice to duplicate the sources via a torrent. This doesn't involve any trust, because the hash of the sources is known beforehand and would allow older nixpkgs to work as long as someone uses the desired package.

[copumpkin]

We've discussed a fetchTorrent before but have mostly resisted because unless we build deep support for it into a persistent Nix daemon, fetchTorrent as we could implement today would be a terrible "torrent citizen", and would never seed.

Probably more promising is to build support for IPFS into Nix itself. See discussion at NixOS/nix#859 and ipfs/notes#51.

Either way, I don't think most of the work would happen inside nixpkgs, so if you think the above issues don't cover what you want, I'd suggest posting more information on the Nix issue tracker rather than in here.

[abbradar]

FWIW I had an experimental fetchTorrent via aria2 but dropped it because of the same reasons -- it'd be an awful "torrent citizen".

[knupfer]

Actually, the ipfs issue seems to be exactly what I wanted, so I'll close this issue.

[Renha]

@abbradar, can you share your fetchTorrent please? I'm currently thinking about packaging the program where using aria2c for install is an official way to install

[milahu]

moved to PR

{
  src = fetchBitTorrent {
    btih = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
    sha256 = "";
  };

pkgs/build-support/fetchbittorrent/default.nix

{ stdenvNoCC
, fetchFromGitHub
, aria2
, cacert
}:

{
  btih,
  sha256 ? null,
  hash ? null,
  name ? null,
  # TODO implement "flat" mode = produce a single file in $out
  outputHashMode ? "recursive", # "recursive" or "flat"
  outputHashAlgo ? "sha256",
  dontUpload ? false,
  # TODO implement partial download, similar to git sparse checkout
  # https://github.com/NixOS/nixpkgs/pull/135881
  #files ? null,
} @ attrs: stdenvNoCC.mkDerivation {
  name = attrs.name or "btih-${btih}";
  trackerlist = (fetchFromGitHub {
    # https://github.com/ngosang/trackerslist
    # using trackers by IP address to avoid DNS blocking
    owner = "ngosang";
    repo = "trackerslist";
    rev = "8ba874e69da0532166644922ea207da3084dff66"; # 2023-02-18
    sha256 = "sha256-QCE7gDtDBLLpFgGWa3r/1YOz16dTXKx6rYy8m0DCH8k=";
  }) + "/trackers_best_ip.txt"; # 16 trackers
  #}) + "/trackers_all_ip.txt"; # 97 trackers
  nativeBuildInputs = [
    # FIXME aria2c has no manpage in nixpkgs
    # https://aria2.github.io/manual/en/html/aria2c.html
    aria2
  ];
  inherit btih outputHashMode outputHashAlgo;
  outputHash = attrs.sha256 or attrs.hash or (throw "sha256 or hash is required");
  buildCommand = ''
    export HOME=$TMP
    mkdir $HOME/.aria2

    # create empty dht.dat file to hide warning
    # TODO file path?
    mkdir -p $HOME/.cache/aria2
    touch $HOME/.cache/aria2/dht.dat
    # /build/.cache/aria2/dht.dat
    # $XDG_CACHE_HOME/aria2/dht.dat
    # $XDG_CACHE_HOME/aria2/dht6.dat

    cat >$HOME/.aria2/aria2.conf <<EOF
    bt-tracker=$(cat $trackerlist | xargs printf "%s,")
    seed-time=0
    ca-certificate=${cacert}/etc/ssl/certs/ca-bundle.crt
    # TODO? use impure env for dht-listen-port
    listen-port=6881
    dht-listen-port=6881
    enable-dht=true
    check-integrity=true
    file-allocation=prealloc
    download-result=hide
    summary-interval=10
    ${if !dontUpload then "" else ''
      # TODO disable seeding completely
      # this will limit upload to 1 byte/sec
      max-upload-limit=1
    ''}
    EOF

    mkdir $out
    cd $out
    echo fetching bittorrent $btih ...
    aria2c "magnet:?xt=urn:btih:$btih"
  '';
}