Glonce

Google DeepMind ships SL2T sign-language-to-text model in Gboard, Live Transcribe

productsThe Glonce editorsGoogle DeepMind🔥80Glonce rating80Impact75Novelty75Relevance100Surprise58Players85Credibility88
Google DeepMind ships SL2T sign-language-to-text model in Gboard, Live Transcribe

Google DeepMind, working jointly with the Android team, introduced sign-language-to-text (SL2T), a massively multilingual translation model, and moved sign language AI out of the lab into a shipping consumer product for the first time: SL2T now powers sign-to-text dictation in Gboard and in Live Transcribe on the Pixel 11, starting with American Sign Language (ASL) to English. The source frames the gap it is closing in scale: AI has advanced rapidly for spoken languages over recent decades, but that progress has not reached the world's more than 200 sign languages or the estimated 70 million Deaf and hard of hearing people who use them. With SL2T, Deaf users can sign to their phone anywhere they would normally type: to search the web, draft messages or documents, or ask Gemini to solve a query or run a task. In Live Transcribe, users can sign their side of a conversation instead of typing back and forth. According to the source's own testers, signing in ASL felt faster, more natural, and more delightful than typing in English.

The source lays out why the problem is hard. Unlike speech transcription, which maps sound to text within one language, sign languages are independent natural languages with their own grammars and lexicons, so translating them requires true machine translation rather than a sequential sign-to-word conversion. The model also has to "see": sign languages carry meaning through simultaneous movement of hands, arms, torso, head and face, which means tracking fine-grained, whole-body motion at high frame rates, a demanding computer vision problem. The source cites this as the reason early approaches such as sign language gloves fell short: sign languages are not simply English performed on the hands.

SL2T was built by combining a culturally informed, user-centred approach with large-scale data. The model trained on more than 100,000 hours of data spanning more than 50 sign languages, with roughly a quarter of that data in ASL specifically. Training jointly across languages, dialects and proficiency levels let the model learn shared underlying structure, and in the source's own experiments this outperformed single-language models. For privacy, SL2T does not process raw video: an on-device model, MediaPipe Holistic, tracks pose landmark locations on the signer, and only those geometric coordinates travel to the server for translation, so the original video can be discarded immediately. SL2T also translates the landmark sequence directly into text, skipping the intermediate "gloss" annotations used in much prior sign-language translation work, which the source says fail to capture non-manual markers and spatial constructions that carry real meaning.

On benchmarks, the source describes SL2T as the most capable sign language translation model to date, citing a zero-shot score of 70 BLEURT on FLEURS-ASL (sd-test), a benchmark for ASL-to-English translation quality, which it says is significantly higher than any previously reported score, though the source does not give a comparison figure for any prior model. Beyond the benchmark, the team says it worked on practical deployment issues: minimizing streaming latency, preventing hallucination on non-signing input, ensuring fairness for the 10% of signers who are left-handed, and improving accuracy for one-handed signing, which is common when a user is holding a phone in the other hand.

The project involved Deaf perspectives at every stage, from conceptualization by Sam Sepah, a Deaf Googler, through data collection with Deaf partners, evaluation in Deaf user studies, and impact assessment with Deaf experts. Google DeepMind also established the AI Sign Language Advisory Committee (AISLAC), bringing in Deaf organizations and subject-matter experts to guide deployment, and co-authored a joint impact report for the SL2T 1.0 release covering the model's capabilities and current limitations, a practice the team says it plans to continue for future sign-language releases. The source describes ASL on Pixel 11 as only the start: the team is working to extend the technology to additional sign languages, to sign language generation, and to broader frontier AI capabilities, and says the Gboard and Live Transcribe features are available at no additional cost.

Key facts

  • SL2T, a Google DeepMind and Android model, now powers sign-to-text dictation in Gboard and Live Transcribe on the Pixel 11, starting with ASL to English; this is the first time sign language AI has shipped in a consumer product.
  • SL2T trained on more than 100,000 hours of data across more than 50 sign languages, with roughly a quarter of that data in ASL, and scores 70 BLEURT zero-shot on the FLEURS-ASL (sd-test) benchmark, which the source calls significantly higher than any previously reported score.
  • For privacy, an on-device model (MediaPipe Holistic) converts the signer's movement into pose landmark coordinates, and only those coordinates, not raw video, are sent to the server for translation.
  • SL2T translates landmark sequences directly into text, bypassing the intermediate "gloss" annotations used in prior sign-language translation work, which the source says lose non-manual markers and spatial meaning.
  • The project was shaped by Deaf perspectives throughout, including conceptualization by Deaf Googler Sam Sepah, and Google DeepMind formed the AI Sign Language Advisory Committee (AISLAC) to guide deployment; more devices and languages are planned.

Why it matters

The source frames this as sign language AI's move from the lab into a real consumer product for the first time, closing a gap it says has persisted even as AI for spoken languages advanced rapidly for decades. That gap is large by the source's own numbers: more than 200 sign languages worldwide and an estimated 70 million Deaf and hard of hearing users who rely on them, none of whom had a sign-language dictation option comparable to voice dictation for hearing users until now.

Who it affects

Deaf and hard of hearing Pixel 11 owners who sign ASL gain a new way to interact with their phone: they can sign instead of type to search, draft messages and documents, ask Gemini to handle a task, or answer in a Live Transcribe conversation. The source also flags who was considered in building it: the 10% of signers who are left-handed, and one-handed signers, who often sign one-handed because their other hand is holding the phone.

How to use it

SL2T is live now in Gboard and Live Transcribe on the Pixel 11 for ASL-to-English sign-to-text dictation, at no additional cost. The source says more devices and additional languages are coming but gives no dates or a list of which devices or languages come next; only ASL-to-English is confirmed as shipped today, even though the training data spans more than 50 sign languages.

How solid is it

This is Google DeepMind's own announcement of its own model, not independent verification. The one concrete external benchmark cited, 70 BLEURT zero-shot on FLEURS-ASL (sd-test), is presented by the source itself, and the claim that this is "significantly higher than any previously reported score" is not accompanied by a comparison number for any earlier model. The development process is unusually well documented for a company blog post: a named Deaf project originator, Deaf user studies, a standing Deaf advisory committee (AISLAC), and a co-authored joint impact report on capabilities and limitations released alongside SL2T 1.0.

Risks and caveats

The source names its own guardrail work, latency, hallucination on non-signing input, left-handed fairness, one-handed signing, without giving figures for how well each is solved, so the practical reliability of the feature outside the source's own testing is not established here. Coverage is also narrower than the training data suggests: SL2T trained on over 50 sign languages, but only ASL-to-English is confirmed live in the shipped product, and the source gives no timeline for when additional sign languages or devices beyond the Pixel 11 will follow.

“According to our testers, signing in ASL is faster, more natural, and more delightful than typing in English.”

— Google DeepMind and Android SL2T team