Low sentence decoding speed

[remyzerems]

Hi,

I'm trying to use STT on Windows 10.

My computer is pretty well sized with an Intel Core i9 7900X, 32Gb of RAM and an NVIDIA GeForce RTX 2080Ti.

I downloaded the Windows native client v1.0.0 unzipped it somewhere.
I got the huge-vocab package and extracted it in the same folder.

Then I ran the following command :
stt.exe --model model.tflite --scorer huge-vocabulary.scorer --audio my_test_wav_file.wav

my_test_wav_file.wav is a 16kHz 16bits mono audio file of 7 seconds duration containing clear speech.

The command takes 26s to decode the sentence !! Means 3.7x...
When running the stt process only uses 24% of the CPU power.

I guess this is not meant to be a normal score, so I'm wondering what I'm doing wrong to get such a poor speed...
What am I doing wrong ??

Thanks for helping me.

[reuben]

Yes that is not normal. Can you verify something for me? What's the size of the model.tflite file?

[remyzerems]

The size of model.tflite is 188 960 976 bytes (180Mb). I got it from here : https://coqui.ai/english/coqui/v1.0.0-huge-vocab

[reuben]

Could you redownload it and try again with the new version? We accidentally released unquantized models, but it was fixed a bit after the release. If you redownload it should be a (much faster) 47MB model.tflite file.

[remyzerems]

Yes !! That's now far more better ! 3.1s ! Thank you for your help !

By the way, I'm wondering about using GPU for decoding. I saw GPU at Coqui level has been removed but I saw that there is the STT_TFLITE_DELEGATE environment variable to set to "gpu".
What is the expected effect of that flag when in Windows ? It seems that it is not applicable on this configuration right ?

[reuben]

Correct, the TFLite GPU delegate is for iOS and Android devices.

[remyzerems]

Cheers reuben! Thanks for that great work!