RealtimeVAD signals end of speech about 100 ms before Silero VAD

RealtimeVAD is an open-source, MIT-licensed Python voice activity detector, currently at alpha version 0.0.5, built for voice assistants. The README's pitch: an assistant can do nothing until it knows the user has stopped talking, because speech recognition, the LLM call and the voice output all wait for that one decision, and a normal detector takes a few hundred milliseconds to make it.
The project keeps Silero VAD to decide what is speech and adds a second, faster check. Every 10 ms it compares the audio level with the room's noise and with the speaker's own level. When the audio goes quiet, an event fires long before Silero's probability has fallen. Input is 16 kHz mono audio in chunks of any size (float32, int16 or int16 bytes). Output is start, pause and end events, plus the audio of the running segment at any time. It runs on one CPU thread, far faster than realtime, and needs only numpy and onnxruntime; the Silero model is bundled, so there is no torch dependency.
The headline numbers come from 3,279 turns from a live voice assistant, each with its trailing silence, at default settings. Speech-off came a median 100 ms before Silero's probability drops, and 206 ms before Silero's VADIterator reports the end. The end event itself comes before Silero's in 82% of turns, by a median 70 ms, and has no gain in the other 18%. That is smaller because end waits 0.24 s after speech-off in both cases. The end comparison uses 2,323 turns where both detectors closed the same segments. The demo video uses a synthetic voice and shows a median lead of 92 ms. Until version 0.0.4 the first benchmark row read 110 ms; the 10 ms went into not cutting off quiet speakers.
The price is stated up front: RealtimeVAD fires in short gaps between words about ten times as often as Silero. The pause event is early and often wrong on purpose. It fires on every breath, short gap between words and stop consonant, so most pause events are followed by more speech. The README says to use pause to start work that can be cancelled (finalize the transcript, call the LLM, prepare the voice output) and cancel when speech continues. Never treat it as the end of the turn; only end closes a segment. If work cannot be cancelled quickly, ignore pause and wait for end, because the false alarms would cost more than they save.
The README also describes integrations. A VADIterator class is a drop-in replacement for Silero's: change the import and end arrives as soon as the audio goes quiet. A LiveKit plugin swaps in for silero.VAD.load(). Against the Silero plugin's defaults, end of speech comes about 380 ms sooner, but most of that is the shorter silence setting (0.25 s instead of 0.55 s), which Silero can be given too. At the same setting RealtimeVAD's end is 65 to 110 ms earlier and LiveKit's turn detector starts 95 to 180 ms earlier. Inside a LiveKit session the turn was committed about 150 ms sooner than with Silero's defaults and about 70 ms sooner than with Silero at 0.25 s.
Version 0.0.5 fixed two problems. First, up to 0.0.4 a single loud 10 ms block after a word (a click, a lip smack, a breath) stopped the quiet count for good and left the end to Silero; that happened in a third of 680 sampled turns, and the count now restarts. Second, the quiet floor used to be fixed at about -54 dBFS, so for quiet speakers such as a phone line or a distant microphone, soft word endings counted as silence and the segment ended while they were still talking. Now the floor is quiet_rms or 20 dB below the speaker's own level, whichever is lower. The authors tested this on the same 3,401 turns, attenuated and sent through a simulated phone line. In their words, quiet speakers are no longer cut, and RealtimeVAD's last end was never later than Silero's in those runs. A noisy quiet line still gets no lead: when noise is within about 20 dB of the speaker, RealtimeVAD ends where Silero does.
An optional normalize_to_dbfs setting (for example -30) amplifies the frames Silero sees by up to 20 dB, never into clipping, until speech sits at that level. It is off by default because it is not free at normal level: 2 of 3,401 turns lost more than 100 ms, and speech-off was earlier than Silero in 96% of turns instead of 98%. For short sounds like "mm-hmm" and "yeah", the README tested 200 real ones from AMI corpus meeting recordings. Up to 0.0.4, 50 were not reported as speech; with the new soft start 25 are missed, 15 of them Silero's own. A start comes a median 160 ms after the word begins.
Key facts
- RealtimeVAD (alpha 0.0.5, MIT) keeps Silero VAD and adds a 10 ms audio-level check against room noise and the speaker's own level, so speech-off is signalled earlier.
- On 3,279 real voice-assistant turns, speech-off came a median 100 ms before Silero's probability drops and 206 ms before Silero's VADIterator reports the end; the end event itself is earlier in 82% of turns, by a median 70 ms.
- The cost: it fires in short gaps between words about ten times as often as Silero, so the pause event must only start cancellable work, never close a turn.
- The lead disappears on noisy quiet lines, the numbers come from one speaker and one capture chain, and input is limited to 16 kHz mono.
- Install with pip install realtimevad; it works as a drop-in for Silero's VADIterator and has a LiveKit plugin.
Why it matters
Every voice assistant waits for one decision before it can respond: has the user stopped talking? Speech recognition, the LLM call and the voice output all sit behind it, and the README says a normal detector needs a few hundred milliseconds to make it. RealtimeVAD attacks that delay by pairing Silero's model with a cheap level check every 10 ms. The reported gain is about 100 ms at the median for speech-off, and the project offers a way to turn it into speed: start work on an early, possibly wrong signal and cancel it if the speaker continues.
Who it affects
Developers building voice assistants and realtime speech pipelines who already use Silero VAD, including those on LiveKit, for whom the project ships a plugin. RealtimeSTT and RealtimeTTS are linked at the top of the README. It matters less if your pipeline cannot cancel work quickly, or if your audio is noisy.
How to use it
Install with pip install realtimevad, then create VAD() and feed it chunks. A start event means speech began and vad.audio() includes about 0.3 s before it; pause means the speaker may be done, so speculate; end carries the finished segment with its audio, start time, end time and the time speech stopped. If the assistant speaks while the microphone is open, pass playback=True for those chunks so its voice is treated as possible echo. To drop it into existing Silero code, change the import to RealtimeVAD's VADIterator. For LiveKit, use realtimevad.VAD.load() in place of silero.VAD.load(). For quiet or phone-line input, VAD(normalize_to_dbfs=-30) is an option, but it costs a little at normal level. Every threshold and duration can be configured, and a demo page runs on your own microphone and shows the lead turn by turn. The code is MIT licensed, and the bundled Silero model is MIT too.
How solid is it
The project is alpha (0.0.5) and the README says the API may still change. All figures are the project's own README measurements, and a benchmark script is provided so you can reproduce them on your own 16 kHz mono recordings. The numbers come from one speaker and one capture chain: browser capture with noise suppression and echo cancellation, mostly quiet rooms. In the 137 noisier turns (room level above -70 dBFS) the median lead was 102 to 124 ms. The first benchmark row read 110 ms until 0.0.4 and is now 100 ms, so the 110 ms in the Hacker News title is the older figure. The LiveKit comparison is also candid that most of the 380 ms gain over the Silero plugin's defaults comes from a shorter silence setting rather than from RealtimeVAD itself. The demo video uses a synthetic voice and shows 92 ms.
Risks and caveats
The pause event is wrong on purpose and fires about ten times as often as Silero in short gaps, so treating it as the end of a turn would cut people off. Earlier is not the same as correct: the README says it did not check against word timings, at the recorded level, whether a speech-off sometimes lands inside a soft word ending. On a noisy quiet line there is no lead, and RealtimeVAD ends where Silero does. Whisper is mostly not detected, while laughter, coughs and sneezes often are. A DC offset in the input hides the silence from the quiet clock, so there is no early end. The soft start still misses 25 of 200 backchannels, and the normalize_to_dbfs option can miss the first quiet words of a stream. The phone-line tests are simulated, and only 16 kHz mono is supported.
“pause is early and often wrong, on purpose.”
— RealtimeVAD README