nixpkgs-update-fetch-github is broken

[zowoq]

cc @ryantm @rhendric

I haven't see this error before:

Exception: ("Got 403, but we can't tell why.", 'Repository access blocked')

We're currently using a branch from my repo for this fetcher. #1160, https://github.com/qowoz/nixpkgs-update-github-releases/compare/master...patches

[zowoq]

Seems that it is the error we'll get when github disables a repo, just need to patch the fetcher to skip it.

❯ curl https://api.github.com/repos/tukaani-project/xz
{
  "message": "Repository access blocked",
  "block": {
    "reason": "tos",
    "created_at": "2024-03-30T00:57:16Z",
    "html_url": "https://github.com/tos"
  }
}

Testing https://github.com/qowoz/nixpkgs-update-github-releases/commit/ad1053ebb8f13e50c12abde967266cc7bdc839d4.

[zowoq]

Fixed in 83c6917.

[zowoq]

Broken again.

Cache dir: /var/cache/nixpkgs-update/fetcher/nixpkgs-update-github-releases
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Host did not send X-RateLimit-Remaining header.
Status code: 200
Cached stats:
{False: 1, True: 9}
Traceback (most recent call last):
  File "/nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/main.py", line 391, in <module>
    main()
  File "/nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/main.py", line 381, in main
    for line in updateLines(meta):
                ^^^^^^^^^^^^^^^^^
  File "/nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/main.py", line 369, in updateLines
    nextVersion = getNextVersion(version, page)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/main.py", line 329, in getNextVersion
    nextVersionDate = latestRelease(*userRepo)
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/main.py", line 243, in latestRelease
    for tag in releases:
               ^^^^^^^^
  File "/nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/main.py", line 225, in iterReleases
    result = getEndpoint(f"/repos/{user}/{repo}/releases?page={page}")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/main.py", line 210, in getEndpoint
    raise Exception(f"No good response after {max_retries} tries")
Exception: No good response after 10 tries

This was a github issue, when using the authenticated api it wasn't setting the correct headers.

[zowoq]

error:
        … while calling the 'listToAttrs' builtin
          at /nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/loadMetaFromPath.nix:42:3:
            41| in
            42|   pkgs.lib.listToAttrs filtered
              |   ^
            43|
        … while calling the 'filter' builtin
          at /nix/store/cpgc7nj9axliidxbqk9wm5kcxv1v4xc0-source/loadMetaFromPath.nix:40:14:
            39|     (pkgs.lib.splitString "\n" (pkgs.lib.fileContents universeFile));
            40|   filtered = pkgs.lib.filter (x: x != null) versions;
              |              ^
            41| in
        (stack trace truncated; use '--show-trace' to show the full, detailed trace)
        error: attribute 'cp313' missing
        at /nix/store/6jgvs97718i2c81f8dyldglpc3hf49xi-source/pkgs/development/python-modules/ray/default.nix:90:14:
            89|       platform = "manylinux2014_x86_64";
            90|       hash = binary-hashes.${pyShortVersion};
              |              ^
            91|     };
        Did you mean one of cp310, cp311 or cp312?

Fixed in NixOS/nixpkgs@e2db3df. Should probably try to handle errors in the fetcher.

[zowoq]

Fetcher has been flaky the last couple of days but it doesn't seem to be a consistent failure.

http 451, deployed a fix. nix-community/nixpkgs-update-github-releases@8677f3c

malt 1.2.3 1.2.4 https://github.com/memtt/malt/releases
mame 0.273 0274 https://github.com/mamedev/mame/releases
mame-tools 0.273 0274 https://github.com/mamedev/mame/releases
mandelbulber 2.32 2.33-dev https://github.com/buddhi1980/mandelbulber2/releases
Endpoint /repos/7USTIN/manga-cli/releases?page=1 not found
Cached stats:
{False: 13161, True: 4760}
Traceback (most recent call last):
  File "/home/zowoq/nixpkgs-update-github-releases/main.py", line 391, in <module>
    main()
  File "/home/zowoq/nixpkgs-update-github-releases/main.py", line 381, in main
    for line in updateLines(meta):
                ^^^^^^^^^^^^^^^^^
  File "/home/zowoq/nixpkgs-update-github-releases/main.py", line 369, in updateLines
    nextVersion = getNextVersion(version, page)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zowoq/nixpkgs-update-github-releases/main.py", line 329, in getNextVersion
    nextVersionDate = latestRelease(*userRepo)
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zowoq/nixpkgs-update-github-releases/main.py", line 244, in latestRelease
    release = tag.get("tag_name")
              ^^^^^^^
AttributeError: 'str' object has no attribute 'get'