Glonce

Phonon on-device TTS ranks top three on Open TTS Leaderboard, authors say

modelsThe Glonce editorsHugging Face🔥59Glonce rating59Impact45Novelty55Relevance95Surprise35Players25Credibility80
Phonon on-device TTS ranks top three on Open TTS Leaderboard, authors say

A post on the Hugging Face blog looks back at the launch of Phonon, which the authors describe as their first open release: a small text-to-speech model that runs on the user's own device. It needs no server and no network, and the typed text stays with the user. The post was written two weeks after the release; the authors say they announced Phonon in April and spent the following six months building on it with partners and early users. That feedback led to smaller models, more languages, and support for custom voices through voice cloning and voice design.

The authors frame the project around a trade-off they say has always been hard: a small model tends not to sound great, and a great-sounding one does not fit on a small device. Phonon is based on the Pocket TTS lineage and uses flow matching to generate audio, which the authors say differs greatly from the techniques in other TTS systems and let them reach this quality in a small model. It also made training hard: it was initially numerically unstable and needed several tricks to give good results in all five languages.

What was released: two model sizes, at about 65 MB and 75 MB, which together speak English, French, German, Spanish and Portuguese. There are more than 100 built-in voices, and users can create their own in Gradium Studio. The authors say Phonon handles things many TTS systems stumble on, such as phone numbers, dates and codes. It runs on phones, laptops, single-board computers and in a browser tab. On licensing, Phonon 31m speaks English and is Apache 2.0, free even in commercial products. Phonon 40m covers all five languages under a non-commercial licence. The other models are CC BY-NC, free for non-commercial projects; for commercial use of those, pricing is based on the number of users, not on how much speech is generated.

Launch demos came from the team and from friends. On an Apple Watch with Wi-Fi off, Phonon talks the user through a coffee brew step by step. On an iPhone with no signal, it counts reps. A robot called Lamp, from Autonomous, wears a sorting hat and talks in character. In a browser, an offline dino runner becomes a radio show hosted by Rex. There is also a sci-fi game voiced on the player's own machine, and Proust read aloud page by page. The Hugging Face team built the Transformers integration so developers could use Phonon there from day one. Developers have since built an ExecuTorch conversion that runs Phonon on a visitor's GPU in the browser, Micdrop for Node.js apps, and Spaces such as smallest-s2s. Phonon was downloaded more than 200,000 times in its first two weeks and is still trending on Hugging Face.

On benchmarks, the authors point to the Open TTS Leaderboard, a new Hugging Face project. With more than 9,000 TTS models on the Hub, it offers one place to compare open models across 23 languages. It scores intelligibility (word error rate), speed (inverse real-time factor and time-to-first audio) and naturalness (UTMOS for English, KOMOS for Korean). Fixed-voice and voice cloning are both evaluated, with speaker similarity added for cloning, and a Listen tab lets people compare outputs and vote. According to the authors, on intelligibility Phonon ranks first in German and Spanish and in the top three in every language they evaluated, next to models up to 100 times its size. On speed, they say the models top the CPU streaming benchmark on time-to-first audio. The post gives no raw metric values.

For inference the authors built a new framework, ptts, specifically for Phonon. It supports more than 10 backends: CPU (Intel, Arm, WebAssembly), GPU (CUDA, Vulkan, Metal, WebGPU) and NPU (Apple Core ML, Snapdragon). Phonon is also in Transformers from v5.21 as the reference PyTorch implementation. Where ptts runs a single stream on device, Transformers targets GPU throughput: batched generation and full-graph torch.compile, for example to synthesise speech datasets at scale. Because Phonon belongs to the Pocket TTS lineage, Pocket TTS was added to Transformers first, and Phonon sits on top as a modular model of about 200 lines describing only what differs.

The post walks through a code example. Phonon generates audio latents autoregressively, each step's output fed back as the next input, so the codec that maps audio to latents plays the role of a tokenizer; Transformers ships it as an audio tokenizer held by the processor. For gradium/phonon-31m-en the example prints 30,788,354 parameters for the model and 10,305,313 for the audio tokenizer. A voice is a tensor of speaker latents, not a recording; the tokenizer ships decoder-only and voices come precomputed in gradium/phonon-voices. The prompt layout ships with the checkpoint as a chat template, so a standard conversation with an empty audio item where the voice goes works across checkpoints. generate returns latents, which the processor decodes to audio and saves as a WAV file.

Looking ahead, the authors say Phonon can speak with any voice: clone one from a short recording with the speaker's consent, or describe a new one in plain text in Gradium Studio, then download the voice file and run it fully on the device. They also train custom Phonon models with design partners: a smaller model for a tight memory budget, Korean, Arabic or Japanese, or fine-tuning on a product's own vocabulary.

Key facts

  • Phonon is a small open text-to-speech model that runs on-device with no server or network; it comes in two sizes, about 65 MB and 75 MB, and covers English, French, German, Spanish and Portuguese.
  • Phonon 31m (English) is Apache 2.0, free for commercial use; Phonon 40m (all five languages) and the other models are CC BY-NC, with commercial pricing based on number of users.
  • The authors report more than 200,000 downloads in the first two weeks, and say Phonon ranks first in German and Spanish on intelligibility and in the top three in every language they evaluated on the Open TTS Leaderboard.
  • They also say the models top the CPU streaming benchmark on time-to-first audio; the post gives no raw metric values.
  • A new framework, ptts, supports more than 10 backends across CPU, GPU and NPU, and Phonon is in Transformers from v5.21 as the reference PyTorch implementation.

Why it matters

Most voice AI makes a round trip to a cloud server for every sentence. Phonon keeps the text on the device and works without a network, and the authors say it does so with a model of only about 65 MB to 75 MB. They attribute the quality to flow matching, inherited from the Pocket TTS lineage, which they call very different from the techniques in other TTS systems. The open weights, including an Apache 2.0 English model, make this usable as a building block rather than a hosted service.

Who it affects

Developers building voice features for phones, laptops, single-board computers or browser apps, especially where users are offline or the text should not leave the device. Teams that need many voices or custom ones are covered too: more than 100 built-in voices, plus cloning and voice design in Gradium Studio. Companies with unusual needs (a smaller model, Korean, Arabic or Japanese, or a product-specific vocabulary) are invited to work with the authors as design partners.

How to use it

Phonon runs in the browser, on phones and on small boards through the ptts framework, or in Hugging Face Transformers from v5.21, where the post's example loads gradium/phonon-31m-en with AutoProcessor and GradiumPhononTTSForConditionalGeneration, builds a chat-style prompt with a precomputed voice from gradium/phonon-voices, calls generate and decodes the latents to audio. Transformers suits GPU throughput work such as building speech datasets; ptts suits a single stream on a device. On licensing, Phonon 31m is Apache 2.0 and free for commercial products. The other models are CC BY-NC, free for non-commercial use; for commercial use of those, pricing depends on the number of users, and the post states no figures.

How solid is it

This is a first-party launch retrospective, published on Hugging Face's site, so the performance claims are the authors' own. The ranks they cite (first in German and Spanish on intelligibility, top three in every language they evaluated, top of the CPU streaming benchmark on time-to-first audio) come from the Open TTS Leaderboard, which the post describes as a Hugging Face project. The post gives no word error rates, latency figures, or names of the models Phonon was compared with. The download count of more than 200,000 is also the authors' figure.

Risks and caveats

The headline results are self-reported and come without numbers, so the margins over other models are unknown. Only the English Phonon 31m is Apache 2.0; the multilingual Phonon 40m and the other models are non-commercial unless the user-based commercial pricing is arranged. The authors also note that training was initially numerically unstable and needed several tricks. Voice cloning in Gradium Studio is described as requiring the speaker's consent. The leaderboard is described as very new, and the ranks depend on the languages and metrics it covers.

“It needs no server and no network, and what you type stays with you.”

— Phonon authors, Hugging Face blog