Add missing packs from 16colo.rs and enable LFS #44
Conversation
|
This diff can't be rendered in the github ui (taking too long, unicorns etc) but I assume this migrates the existing library to lfs? checking out this branch locally atm results in an lfs quota error which is already problematic unfortunately. the resulting state is a broken local working copy of master can repair this state with i ran into similar quota issues and looked at ipfs to backend lfs, you can add alternate storage solutions to backend lfs but these all add additional complexity. My notes from back when i tested the ipfs backend are here: https://github.com/sairuk/ipfs-lfs-test just to demonstrate an option |
|
This seems like it's back to the territory of "just leave it as is"
…On Tue, Nov 8, 2022, 1:31 PM sairuk ***@***.***> wrote:
This diff can't be rendered in the github ui (taking too long, unicorns
etc) but I assume this migrates the existing library to lfs?
checking out this branch locally atm results in an lfs quota error which
is already problematic unfortunately.
$ git checkout enable-lfs
Updating files: 100% (5661/5661), done.
Downloading 1990/acdu1190.zip (91 KB)
Error downloading object: 1990/acdu1190.zip (85f7235): Smudge error: Error downloading 1990/acdu1190.zip (85f7235f99b4b6b2a72cad8c160579acde00624a768e57df95360536dbbafd7c): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
the resulting state is a broken local working copy of master
$ git checkout enable-lfs
error: The following untracked working tree files would be overwritten by checkout:
.gitattributes
1994/ac!-05l.lha
1994/coopcol2.lha
1994/ep-bcsfb.lha
1995/1oo!-ger.lha
1995/al-cohol.lha
1995/avl-clip.lha
1996/1oo-answ.lha
1996/1oo_gnor.lha
1996/con-fasc.lha
1996/m's-odds.lha
1997/azki4all.lha
1997/dgs-new!.lha
1997/dkb_egs.lha
1997/dkb_zack.lha
1998/release.nfo
2004/acid-100.nfo
2016/BLNDR2016.ZIP
2016/TRACKERFIX_2016.ZIP
Please move or remove them before you switch branches.
Aborting
can repair this state with git clean -fdx (force/+directoriess/+ignored)
but yeh unfortunately the enable-lfs branch still cannot be checked out
from this repo at the moment
i ran into similar quota issues and looked at ipfs to backend lfs, you can
add alternate storage solutions to backend lfs but these all add additional
complexity. My notes from back when i tested the ipfs backend are here:
https://github.com/sairuk/ipfs-lfs-test just to demonstrate an option
—
Reply to this email directly, view it on GitHub
<#44 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAEMH3M2JLSLJ3LAJUOFTTWHKMBPANCNFSM6AAAAAARVTS3UU>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
yeh seems so, what was the motivation to go to lfs? lfs max unpaid size is only 1GB. Is the repo topping out or are the packs getting too large to host here in a normal git repo? Can this repo get an exception from github to enable this historical (+moddern) collection to be hosted free of charge? I've read elsewhere of a couple of other projects (historical/emulation) approached github (maybe before the ms acquisition) and have some kind of deal in place. |
|
@lordscarlet @sairuk ... didn't see this when I created my individual PRs, was getting an error trying to do it all in a single request... I didn't do anything with LFS on my separate PRs. |
|
Someone mentioned they were having trouble downloading the archive, and had
written some sort of script to get it to work. So I thought I would try
LFS. It definitely doesn't need it, but I thought it might help some people.
…On Tue, Nov 8, 2022 at 10:11 PM sairuk ***@***.***> wrote:
yeh seems so, what was the motivation to go to lfs? lfs max unpaid size is
only 1GB. Is the repo topping out or are the packs getting too large to
host here in a normal git repo?
Can this repo get an exception from github to enable this historical
(+moddern) collection to be hosted free of charge? I've read elsewhere of a
couple of other projects (historical/emulation) approached github (maybe
before the ms acquisition) and have some kind of deal in place.
—
Reply to this email directly, view it on GitHub
<#44 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAEMH27RZ6C7ZOEXRKEWYLWHMI73ANCNFSM6AAAAAARVTS3UU>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
I think I need to do a new PR without LFS
…On Wed, Nov 16, 2022 at 4:50 PM Michael J. Ryan ***@***.***> wrote:
@lordscarlet <https://github.com/lordscarlet> @sairuk
<https://github.com/sairuk> ... didn't see this when I created my
individual PRs, was getting an error trying to do it all in a single
request... I didn't do anything with LFS on my separate PRs.
—
Reply to this email directly, view it on GitHub
<#44 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAEMHZTDTC7M7OLTJT4LLDWIVJLDANCNFSM6AAAAAARVTS3UU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I also tried to convert the archive to use LFS at some point including rewriting all the commit history, and I now regret not having taken notes about the process... But basically it didn't gain us much at all regarding storage space because the packs do not change. So it's not worth it, but we should probably document the details so we don't end up trying to revisit this decision every couple of years or so. |
|
@lordscarlet Yes, it would be nice if you could do a new PR without LFS, and possibly separate the commits per year or decades like @tracker1 did? Do we want to add the MIST packs though? I have carefully been avoiding adding them while I was still updating the archive on a regular basis: this is an ANSI/ASCII Art Archive after all :-) |
|
Sorry, that reminds me, need to update (still) to get actual zip files, where it looked like it snagged a couple html results in them. Though my separation was mostly that I couldn't push a larger commit to github. |
I apologize for not doing these in separate commits, but: