Entropy coding test

[tansy]

Entropy coding test

Inspired by this issue, and also this QOI issue I did some test of 'entropy coding' for qoi (compressing with regular compressors).
Used subset from benchmark images and default compression levels (except zstd, which was set to 12, to somewhat match gzip level 6.
Scripts used to make this test, in case someone wanted to test it themself are also attached.

That's the results: (gz-H is pure Huffman from deflate (`pigz -H`)), rans_jks is jkbonfield's rans_static)

20737618 bmp
 7035249 qoi
 6630663 qoi.lz4
 5881278 qoi.rans_jkb
 5850003 qoi.gz-H
 5721098 png
 5476417 qoi.zst
 5471989 qoi.gz
 5113443 qoi.lzma
 5102454 qoi.bz2

sorted by ratio

method	ctime	cspeed	dtime	dspeed	compr.	ratio
qoi.bz2	2.762	7.5082	1.491	13.9085	5102454	0.246048
qoi.lbz2	2.124	9.7634	1.122	18.4827	5102454	0.246048
qoi.lzma	5.934	3.4947	1.534	13.5187	5113443	0.246578
qoi.gz	1.410	14.7075	0.543	38.1908	5471989	0.263868
qoi.zst	2.061	10.0619	0.491	42.2355	5476417	0.264081
png	10.308	2.0118	2.361	8.7834	5721098	0.275880
qoi.gz-H	0.859	24.1416	0.549	37.7734	5850003	0.282096
qoi.rans_jkb	0.575	36.0654	0.448	46.2893	5881278	0.283604
qoi.lz4	0.533	38.9074	0.409	50.7032	6630663	0.319741
qoi	0.437	47.4545	0.332	62.4627	7035249	0.339251

sorted by ctime

method	ctime	cspeed	dtime	dspeed	compr.	ratio
qoi	0.437	47.4545	0.332	62.4627	7035249	0.339251
qoi.lz4	0.533	38.9074	0.409	50.7032	6630663	0.319741
qoi.rans_jkb	0.575	36.0654	0.448	46.2893	5881278	0.283604
qoi.gz-H	0.859	24.1416	0.549	37.7734	5850003	0.282096
qoi.gz	1.410	14.7075	0.543	38.1908	5471989	0.263868
qoi.zst	2.061	10.0619	0.491	42.2355	5476417	0.264081
qoi.lbz2	2.124	9.7635	1.122	18.4827	5102454	0.246048
qoi.bz2	2.762	7.5082	1.491	13.9085	5102454	0.246048
qoi.lzma	5.934	3.4947	1.534	13.5187	5113443	0.246578
png	10.308	2.0118	2.361	8.7834	5721098	0.275880

sorted by dtime

method	ctime	cspeed	dtime	dspeed	compr.	ratio
qoi	0.437	47.4545	0.332	62.4627	7035249	0.339251
qoi.lz4	0.533	38.9074	0.409	50.7032	6630663	0.319741
qoi.rans_jkb	0.575	36.0654	0.448	46.2893	5881278	0.283604
qoi.zst	2.061	10.0619	0.491	42.2355	5476417	0.264081
qoi.gz	1.410	14.7075	0.543	38.1908	5471989	0.263868
qoi.gz-H	0.859	24.1416	0.549	37.7734	5850003	0.282096
qoi.lbz2	2.124	9.7635	1.122	18.4827	5102454	0.246048
qoi.bz2	2.762	7.5082	1.491	13.9085	5102454	0.246048
qoi.lzma	5.934	3.4947	1.534	13.5187	5113443	0.246578
png	10.308	2.0118	2.361	8.7834	5721098	0.275880

As can be seen lz4 is inefficient in this application, which becomes clear when one realises that lz4 does not use any entropy coding, and the entropy coding, even pure prefix code like huffman from deflate, is what makes a difference in this contest. Lzma is very tight but still loses to bzip2, both in terms of compression and speed. Zstd turns out to be worse and slower than gzip, only decompression is slightly faster but it doesn't make up to it.
RANS static is on par with huffman compression though it's faster, specially with compression.

All in all, the best compressor (by ration) is bzip2, and by speed gzip/deflate, which beats png both, with compression (4%-5%) and with speed (~7 times).

---

qoiformat-bench-1b.zip

[chocolate42]

You might want to try disabling qoi's index op entirely in the presence of external compression, from what I recall it's often detrimental to space-efficiency as it gets in the way of the compressor.

[tansy]

You might want to try disabling qoi's index op

Don't know what you're taking about.

I used fast and already available tools, that are, at least some, part of regular browsers (gzip, gzip's huffman). They don't mean to be super 'tight' but, rather, fast.

[chocolate42]

I mean that qoi's index op, QOI_OP_INDEX is bad for the overall filesize when reasonable compression like zstd is used on a qoi file. You're probably better off disabling the index op, which still results in a completely valid qoi file. Here's a quick example:

#43 (comment)

[tansy]

I will check it.
One thing is apparent here - lz4, which you used there, is not good compressing qoi processed/compressed images. Only entropy coding, even simple, pure prefix/huffman is efficient with it.