Fix no_speech_prob calculation - #3945
Open
mculbert wants to merge 2 commits into
Open
Conversation
Logits for SOT were not set correctly for the no_speech_prob calculation, because wisper_decode_internal() copies computed logits only if batch.logits[i] is true, so set batch.logits[sot_index] = 1. The sot_index is the token after the prompt prefix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
no_speech_probis always ≈0 becausewhisper_decode_internal()only copies the computed logits whenbatch.logits[i]true (whisper.cpp:2951), which is not the case for the SOT position. Moreover,no_speech_probis calculated from position 0 (state->logits, whisper.cpp:7179-7180), but if a prompt prefix is supplied, the SOT is actually after the prefix, so the wrong position is decoded as the SOT forno_speech_prob.This appears to be the root cause of issue #1881.
Fix
Set mask true for the SOT position before calling
whisper_decode_internal()inwhisper_full_with_state(). The SOT position is the next after the prompt prefix, if any. Then, calculateno_speech_probfrom the computed SOT index based on the actual prefix, rather than always from 0.Verification
Using the fixture from #1881, before the fix:
yields:
And the
no_speech_prob(probe with a temporaryfprint()where it is calculated) is 2.71956219e-06 ≈ 0.After the fix, the same command yields
no_speech_prob = 0.905007958, correctly identifying that no speech is present, and the hallucinated "[музыка]" is suppressed.AI disclosure
This issue was identified by Claude Opus 4.8 as the root cause of a bug in a downstream project that depends on this calculation. I have manually verified the diagnosis and the fix that Claude proposed, and I wrote this PR description myself.