Files in manifest

[ArneBab]

This changes freesite inserts to put the smallest files into a container, up to 2 MiB.

It starts with the smallest files referenced from index.html and includes all files which fit in 2 MiB. Then it checks whether further unreferenced files fit into 2 MiB. Then it adds all further files, starting with the slowest, until there are at most 500 files left out of the container. The rest of the files are uploaded as separate files as before (CHKs).

For graphic-heavy freesites like Freenet_Meltdown, this can reduce the site load time by more than factor 3.

[ArneBab]

the current last comment hopefully fixes a regression I did not realize was there: freesitemgr never marked a site as done.

[ArneBab]

there’s still a bug which leads to freesitemgr seeing some sites as changed which did not change.

[ArneBab]

Commit ef5325e finally fixes the mistaken reupload. With this, I consider this ready for merging. There are still bugs, but not more than before adding files to the manifest.

And the changes are no longer a privacy risk (uploading many sites at the same time even though they did not change would allow timing attacks to correlate IDs).

[Thynix]

It looks like this library assumes Python 2 in the way it imports things, and so currently fails to run if Python 3 is the default Python. (As it is on Arch.) That's a separate problem though.

[ArneBab]

ah, yes - for starters I think I should just use python2 as binary. I consider porting to python3 as important, but not blocking (it’s no breakage, but rather not adding a new feature).