Fixed VAD to work when using whisper_full_with_state

[mdestagnol]

Currently whisper_full_with_state doesn't work with VAD.

The main inference method is whisper_full_with_state. This method takes a whisper_context and a whisper_state. On top of this method are built 2 convenience methods, whisper_full and whisper_full_parallel.

The VAD logic is currently only implemented in those convenience methods, but not in whisper_full_with_state itself. This is an issue whenever we need to deal with more than one state (when processing multiple real time streams for example).

This pull request moves the logic into whisper_full_with_state and fix a few incorrect calls to ctx->state instead of state directly.

[danbev]

@mdestagnol Thanks for the pull request.

I've been thinking about this and if I recall correctly, the original pr for VAD actually did something like this. I recall getting feedback about the state should be passed in and not be updated inside of this function (I've not been able to find the comment though I'm afraid).

So I think exposing whisper_vad might be a better option here.

[mdestagnol]

@mdestagnol Thanks for the pull request.

I've been thinking about this and if I recall correctly, the original pr for VAD actually did something like this. I recall getting feedback about the state should be passed in and not be updated inside of this function (I've not been able to find the comment though I'm afraid).

So I think exposing whisper_vad might be a better option here.

Isn't state's purpose to persist the context of whisper_full? State is how the whisper_full_with_state method maintains its own context between different runs. It is actively modified by whisper_full_with_state at each run (even outside of VAD). What do you think?

[tazz4843]

Whatever route is taken, this should be documented anyways since it was extremely confusing wondering why VAD wasn't working despite having the switch on.

[danbev]

What do you think?

Sorry, I make a mistake (recalled incorrectly) about the reason for moving the VAD processing out of whisper_full_with_state. Looking at it closer I found this was moved in 7497754 and related to when parallel processing in use (-p/--processors). We would have to take the parallel processing into account before moving this back into whisper_full_with_state or that parallel processing will not work correctly.