16 kHz required instead of 16 bit (main example)

[rubendax]

In the main example, audio files must be at an audio sampling rate of 16 kHz to be accepted:
read_wav: WAV file 'xyz.wav' must be 16 kHz.

However, in the readme it says:
"Note that the main example currently runs only with 16-bit WAV files".
The readme also suggests using ffmeg to convert the file to 16 bit with the -ar 16000 option. However, -ar controls the audio sampling frequency in hertz, not the audio bit rate.

Having a 16 bit depth requirement makes sense as it is the standard for recorded audio, however, 16 kHz sampling rate is far from standard as the vast majority of recorded audio is at 44.1 kHz (44100 hz) sampling rate or higher.

I could be misunderstanding the intention here, it is totally possible that converting to a lower sampling rate is needed to decrease the overhead for whisper.cpp. However, based on the readme, it seems that there might be some confusion between audio sampling rate and audio bit depth, so I thought I should mention it, just in case.

---

As I side note I would like to give a huge shoutout to ggerganov. Your work has been invaluable for the open source ML/AI community and has completely changed the landscape. Thank you.

[ggerganov]

The 16 kHz requirement is from the Whisper model: https://github.com/openai/whisper/blob/248b6cb124225dd263bb9bd32d060b6517e067f8/whisper/audio.py#L13

The 16-bit requirements is from whisper.cpp as it is easier to support just a single data format.
But in theory, you can modify main.cpp to load any format. All you have to do is convert it to 32-bit Float PCM before passing it to whisper:

https://github.com/ggerganov/whisper.cpp/blob/2b6a0743054a54b1bfb2204a655fc2910cdc240d/examples/common.h#L99-L106

[rubendax]

Thanks so much for your response. Apologies that I neglected to check the original whisper model for the 16kHz sample rate requirement.

[CreativeSelf0]

How can I support bit depth 32?