Heartbeat Thinking lets voice agents run tools while you speak
A preprint introduces Heartbeat Thinking, a method for tool-using realtime voice agents. The problem it targets: such agents often leave users waiting after they stop speaking, even when the information needed to act arrived earlier in the request.
Speculative approaches already exist. They begin work during speech. But the paper says that chunk-bounded or interrupt-on-update generation can truncate a tool call before its arguments are complete. That ties computation time to speech updates and limits how much useful work gets finished in advance.
Heartbeat Thinking handles updated input differently. When the speech input changes, it starts new Thinker invocations, but it lets ongoing invocations and already dispatched tools run to completion instead of cutting them off. Periodic invocations read the latest partial transcript together with a shared record of completed reasoning and tool results. Those results help fill in missing arguments and start dependent calls, so preparation can progress across successive speech updates.
The authors test on three benchmarks: SpeechFCEval, FDBench v3 and AudioCRAG. Against a SHANKS-style windowed baseline, they report that Heartbeat improves aggregate speculative tool coverage or retrieval reuse, with effects on final task quality that depend on the benchmark. The headline numbers come from FDBench v3, where Heartbeat increases tool coverage by 16 percentage points and reduces mean final-response latency by 36.4%.
The authors conclude that preserving ongoing computation during speech can turn listening time into reusable preparation for faster and more capable voice-agent responses.
Key facts
- Heartbeat Thinking starts reasoning and tool calls while the user is still speaking, rather than waiting for speech to end.
- New Thinker invocations handle updated input, while ongoing invocations and dispatched tools are allowed to finish instead of being interrupted.
- On FDBench v3, tool coverage rises by 16 percentage points and mean final-response latency falls by 36.4%, versus a SHANKS-style windowed baseline.
- Evaluated on SpeechFCEval, FDBench v3 and AudioCRAG; effects on final task quality are described as benchmark-dependent.
Why it matters
Voice agents that call tools often make people wait after they finish speaking, even when the key details came early in the request. Earlier speculative methods try to use that time but can cut a tool call off before its arguments are complete. Heartbeat Thinking is aimed at exactly that gap: it keeps computation running across speech updates so listening time becomes preparation.
Who it affects
Teams building realtime, tool-using voice agents, and the people who use them, mainly through how long they wait for a final response. The paper frames the work around voice agents that call tools or retrieve information during a request.
How to use it
The text gives no code, model names or availability details, so there is nothing to download or run from this summary. The design idea is the usable part: let new invocations read the latest partial transcript plus a shared record of finished reasoning and tool results, and do not abort calls already in flight when the transcript changes.
How solid is it
This is a preprint reporting the authors' own results on three benchmarks: SpeechFCEval, FDBench v3 and AudioCRAG. The 16 percentage point and 36.4% figures are for FDBench v3 only, against a SHANKS-style windowed baseline. The 36.4% is a relative reduction in mean final-response latency, and no absolute latency or baseline coverage figures are given.
Risks and caveats
Effects on final task quality are described only as benchmark-dependent, with no direction or size stated, so faster responses may not mean better answers everywhere. Gains on SpeechFCEval and AudioCRAG are described only in aggregate terms (tool coverage or retrieval reuse), with no per-benchmark numbers. The comparison is to one baseline style, so wider claims should wait for independent testing.