simplify (and optimize) decode_size_with_offset #1904

Job	Run time
Checks	53s
Wheel on macos-12 py-3.8	2m 20s
Wheel on ubuntu-latest py-3.8	2m 54s
coverage	8m 28s
Wheel on windows-latest py-3.8	2m 38s
Run fuzz targets (macos-12, 3.8)	16m 3s
Unit tests (macos-12, 3.8)	5m 5s
Run fuzz targets (ubuntu-latest, 3.8)	12m 56s
Unit tests (ubuntu-latest, 3.8)	3m 30s
Unit tests (windows-latest, 3.8)	6m 33s
Upload to PyPI	30s
	1h 1m 50s

Provide feedback