Unicode Error for Hindi transcription

[rahulshivajipawar]

When doing transcription in Hindi for a file, I encounter invalid unicode character.

I have noticed this with many Hindi files.

Used whisper-large-v2 mode for inference on CPU. Have noticed the same issue when inferencing on GPU as well.

I am guessing the issue is: whisper model token output (BPE encoded) is not getting correctly mapped to unicode characters.

[rahulshivajipawar]

This is an example of a file on which it fails - https://drive.google.com/file/d/1_BFuNOAqM3yv4P2A0i8KOT_RZ6LnYSCt/view?usp=sharing

[bobqianic]

I'll explore this further. There might be an issue with the tokenizer.
Take a look at this discussion for more insight: #1313 (comment)

[rahulshivajipawar]

I'll explore this further. There might be an issue with the tokenizer. Take a look at this discussion for more insight: #1313 (comment)

Looked at the issue you referenced, looks like it is similar to this #1313 (comment)

[rahulshivajipawar]

Another side observation - I have found the accuracy for Hindi transcription via whisper.cpp to be much lower than when using openai API directly.

[bobqianic]

Another side observation - I have found the accuracy for Hindi transcription via whisper.cpp to be much lower than when using openai API directly.

Agreed. You don't even need to use their API. Just by using pip install openai-whisper and running large-v2 locally, you can achieve better results than whisper.cpp in most languages apart from English.