Glonce

OpenAI details GPT-Live, its full-duplex voice architecture

infrastructureThe Glonce editorsOpenAI🔥76Glonce rating76Impact75Novelty65Relevance95Surprise50Players85Credibility90
OpenAI details GPT-Live, its full-duplex voice architecture

OpenAI engineers Justin Uberti and Zahan Malkani published a technical account of GPT-Live, described as OpenAI's third-generation voice AI system. Earlier voice systems relied on a small turn-detector model to guess when the user had finished speaking before the large language model could respond: guess too soon and the user gets cut off, guess too late and the reply feels sluggish. GPT-Live removes the turn detector from the audio path entirely. Its voice model is full-duplex, meaning it can listen and speak at the same time, so conversation no longer depends on a separate decision about whose turn it is. When a query needs deeper reasoning or tool use, GPT-Live can consult OpenAI's frontier models, such as GPT-5.5, without interrupting the flow of the conversation; that delegation runs on a separate asynchronous path so a slow tool call cannot stall the audio.

Building this took OpenAI about six months, during which the team reworked model inference, context management, and media transport. The media frontend and inference logic were rewritten in Go, replacing a previous Python asyncio implementation; the authors say this significantly improved the smoothness of frame delivery, to the point that the new system's p95 latency matches the old system's p50 (no absolute millisecond figures are given). Transport runs over WebRTC, which the team says can subtly stretch or accelerate audio playback to absorb packet loss and clock drift without introducing gaps. To cut startup delay, OpenAI built what it calls the WebRTC Abridged Roundtrip Protocol (WARP), which the post says reduces media and data startup from six network round trips down to one.

On the inference side, the system uses a seamless handoff mechanism to move a live session between model instances: a replacement instance is warmed up, prefilled with the current conversation context, run in parallel with the existing one, and cut over once ready. The same mechanism is reused for dynamic context compaction, so a long-running call can have its context trimmed to fit the model's limit without a media interruption, even though compaction normally invalidates the model's key-value cache and would otherwise require a fresh, slower prefill. The application server also reconstructs the continuous audio stream into discrete user and assistant messages for parts of the product, such as ChatGPT's conversation UI and its analytics and safety infrastructure, that still expect turn-based text; it keeps a speculative, updatable view of the conversation for the UI and a separate finalized transcript for logging.

The authors say this architecture already powers a growing range of ChatGPT Voice capabilities, including a newly launched ability to let ChatGPT Voice control the user's computer and coordinate the user's agents in the ChatGPT desktop app. No launch date or version number is given for that capability, and the source text supplied for this story cuts off mid-sentence partway through the description of WARP's protocol-level improvements, so further detail on WARP beyond the six-to-one round-trip reduction was not available.

Key facts

  • GPT-Live is OpenAI's third-generation voice system; it removes the separate turn-detector model and is full-duplex, letting it listen and speak simultaneously.
  • The rebuild took about six months and included rewriting the media frontend and inference logic in Go instead of Python asyncio, matching the new system's p95 latency to the old system's p50.
  • GPT-Live can delegate deeper reasoning or tool use to frontier models such as GPT-5.5 on a separate asynchronous path, without interrupting the live conversation.
  • OpenAI built a new protocol, WARP, that cuts WebRTC media and data startup from six network round trips to one.
  • The architecture already powers ChatGPT Voice, including a newly launched feature that lets it control the user's computer and coordinate agents in the ChatGPT desktop app.

Why it matters

Voice AI has long been bottlenecked by turn detection: a small model has to guess, in real time, when a speaker is done, and getting it wrong either cuts the user off or makes the assistant feel slow. GPT-Live sidesteps that guess by making the voice model full-duplex, listening and speaking at once, and by moving deeper reasoning onto a separate path that runs frontier models like GPT-5.5 without blocking the live audio. That is a structural change in how conversational AI is built, not just a latency tweak.

Who it affects

Directly, this is a description of internal OpenAI engineering, but the effects reach ChatGPT Voice users, since the architecture already powers that product, including its newly launched ability to control a user's computer and coordinate the user's agents inside the ChatGPT desktop app. It is also relevant to engineers building similar real-time voice or streaming systems, since the post details concrete techniques such as stateful model handoffs and a new WebRTC startup protocol.

How to use it

GPT-Live is not a separate product a developer buys or configures; it is the underlying voice architecture inside ChatGPT Voice. The practical entry point for most readers is ChatGPT Voice itself, including the newly launched computer-control and agent-coordination capability in the ChatGPT desktop app. The post gives no pricing, availability, or API details for GPT-Live as a standalone offering.

How solid is it

This is a first-person technical account written by two OpenAI staff engineers (Justin Uberti and Zahan Malkani, Members of Technical Staff) and published on OpenAI's own site, so it is a primary source for what the company says it built, not an independent evaluation. The specifics are concrete: a six-month build timeline, a Go rewrite of the media frontend, a WebRTC-based transport with the new WARP protocol cutting round trips from six to one, and a p95-matches-old-p50 latency claim. No absolute latency numbers (milliseconds) back that comparison, and no independent benchmark or third-party test is cited.

Risks and caveats

As a self-published engineering post, the claims have not been independently verified. The latency improvement is stated only as a percentile comparison without absolute figures, so it cannot be directly compared to competing voice systems. The source text available for this story was truncated mid-sentence during the description of WARP's protocol-level optimizations, so some detail on how WARP achieves its round-trip reduction, beyond the six-to-one figure, is not captured here. No date is given for GPT-Live's rollout or for the newly launched desktop capability.