Error tolerance control

[m13253]

Is it necessary to have an error tolerance control mechanism?

I have to sides of view:

Positive:

• Data should be transported reliably and integrity be kept
• Corrupted data causes noise on decoding or even decode failure

Negative:

• Error tolerance requires more data rate (50% in estimate), which is insufficient
• Integrity can be built on top of transport layer, following the example of OSI model
• Lossy media file does not require much data integrity
• Different algorithms can be chosen, e.g. Hamming Code, to keep data integrity, of which the decision depends on different types of file
• Transport error, alongside with rounding error, can be prevent by choosing moderated or conservative parameters

Maybe there should be another Transmission Control Layer built on top of LVDO Transport Layer, with advanced data rate control, framerate control, Youku watermark bypass solution, and H.264 motion estimate bypass solution.

[m13253]

Submit your opinions please, @jabbany

[jabbany]

Yes, I think some error checking is probably useful. The noise is too high with default settings and gets really high if transcoded. We should probably at least know how much data loss we will get (guaranteed throughput).

[m13253]

Guaranteed data loss is unable to get.

However lvdoenc provides how many bytes are there in a frame. It depends on your current settings.

./lvdoenc -s 640x480 -b 8 --qmin 1 --qmax 4
lvdo: [info] bytes per frame: 10800

Errors are from two aspects:

1. Rounding error: It happens during DCT and quantization.
2. Transcoding error: It happens during H.264 (or other thing) encoding.

LVDO can only control rounding error. It can not control transcoding error.
However, when you see [warning] clipping xxx to xxx, you know something is wrong.

[m13253]

可以看一下类似 WinRAR “自我修复”的功能。
现代无线电通信也在使用这样的功能。

那就是，100% 的数据包，成功接收到其中任意 90% 的数据包（这个阈值可控），就可以修复损坏的数据，并且解码出全部的数据。

如果你感兴趣可以试着实现它。

[Skylion007]

I actually did implement this feature in Python that links to my LVDO Windows fork. The algorithm I used is Reed-Solomon encoding and it worked quite well. I believe that is what most radio operators use. While I don't understand all the math behind it, you can find several implementations of it online.

[felixonmars]

IIRC Reed-Solomon was used in the Parchive project, and libpar2 should be available in most distro.

[Skylion007]

Yeah I actually used Par archives before switching over to Reed-Solomon. There are tons of libraries for the algorithm written in C as well such as https://github.com/catid/longhair . Googling will yield a lot of results. The main issue is just integrating it into the repo.

[m13253]

I actually did implement this feature in Python that links to my LVDO Windows fork.

I am very happy to see friends from all over the world who pay attention to my experiment.
LVDO was originally written as a hobby project, to state against the increasingly strict censorship of online videos conducted by Chinese government. (By the time this repo was started, a Chinese video site named Sina Video was shut down, just because the web master did not delete a porn video in time before the MOC found it)
It is unsafe for me to write these code, also I do not have enough time to work on it. So I will appreciate it if anyone could continue this experiment.