Why run the example of "Single Audio Stream" slow ?

[garymmi]

Example of Single Audio Stream take 7.769s

wav = f'{files_dir}/en.wav'

for batch in single_audio_stream(model, wav):
    if batch:
        print(batch)

but speed is slow compared with the example of "Full Audio" (take 2.879s)

[snakers4]

Can you quantify "slow"?

When you just process the whole audio, you obviously assume that you have this audio on disk on in memory (or wherever).
Therefore you get tremendous boosts from batching. If you process files on disk - there is no point in streaming in case of PyTorch NNs.

When you do streaming, you pretend to "listen" the audio one chunk at a time like in real-life when you listen to someone.
Anyway it ends up working much more fast than real-time because in the example the python iterator does not really "wait" for audio to play, but processing audio one chunk at a time is obviously slower than just batching several chunks together.

If the speed is not enough for your applications, try a 10k model, it is 3-4x faster on server hardware and 100x smaller.

We also did some research on model speed, when you mimic streaming it takes:

• Around 3-4ms for a small model per 300ms audio batch (each batch consists of 8 windows, so you may assume that 1 window "takes" around 0.5 ms);
• One 30 ms window in webrtc takes around 0.05 ms, so a 300ms window would be around 0.5 ms;
• We tried going even smaller (1k params) but speed boosts are negligible and quality starts to drop off even more;
• We are already on the same order of magnitude with WebRTC and I doubt you can go much faster;

[snakers4]

but speed is slow compared with the example of "Full Audio" (take 2.879s)

This is roughly in line with our single thread benchmarks (since this file is 60 seconds long), assuming you use num_steps=8 (also maybe some initial warm-up time)

We have decided not to publish the GPU version (because it has very little production use), but if your target is processing files on disk, you can run 10 1-thread processes in parallel and you will get 300-400 RTS, which is ample

[garymmi]

how to try 10k model
i use example code to download model
model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',
model='silero_vad',
force_reload=True)

[snakers4]

how to try 10k model

https://github.com/snakers4/silero-vad#getting-started

model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',
                              model='silero_vad_micro',
                              force_reload=True)