Glonce

ScriptMoE brings script-aware mixture-of-experts to multilingual scene text recognition

researchThe Glonce editorsHugging Face🔥70Glonce rating70Impact65Novelty75Relevance90Surprise45Players35Credibility85
ScriptMoE brings script-aware mixture-of-experts to multilingual scene text recognition

Researchers describe two linked contributions aimed at multilingual scene text recognition (STR), the task of reading text embedded in photos across many languages and writing systems. The first is TextMuSS-10M, a large-scale synthetic dataset spanning 10 scripts and 229 languages, built to supply balanced training data for languages where real labeled scene text is scarce. The second is ScriptMoE, a script-aware Mixture-of-Experts architecture. The authors frame the problem as a choice between two flawed approaches: deploying a separate recognizer for each language, which raises cost and lets errors accumulate across a pipeline, or relying on large vision-language models (VLMs), which are expensive to run and still inaccurate on many scripts. ScriptMoE is designed to be simpler than the per-language approach, lighter than VLMs, and more accurate than both. Architecturally, it shares a single visual encoder across all scripts and replaces the usual dense decoder with a sparse MoE block: an image-level router sends each image to its top-2 script-aligned experts, while a separate shared expert absorbs knowledge that transfers across scripts. The authors also assembled TextMuSS-Bench, an evaluation benchmark covering the same 10 scripts across 10,899 images. On this benchmark, ScriptMoE reaches 82.06% accuracy, beating the strongest STR baseline by 1.31 percentage points. In an end-to-end test, the authors took PP-OCRv5, an existing OCR pipeline, and swapped in ScriptMoE as its recognizer component while leaving the rest of the pipeline unchanged; on the CC-OCR end-to-end multilingual task this lifted the F1 score from 65.71% to 80.89%, slightly ahead of the best vision-language model tested, which scored 80.73%, while ScriptMoE uses only a fraction of that VLM's parameter count.

Key facts

  • TextMuSS-10M is a synthetic scene text dataset spanning 10 scripts and 229 languages, built to cover languages where real training data is scarce
  • ScriptMoE shares one visual encoder and swaps the dense decoder for a sparse Mixture-of-Experts block: an image-level router picks the top-2 script-aligned experts per image, plus one shared expert for cross-script knowledge
  • On the authors' TextMuSS-Bench (10 scripts, 10,899 images), ScriptMoE hits 82.06% accuracy, 1.31 percentage points above the strongest STR baseline
  • Replacing only the recognizer in the PP-OCRv5 pipeline with ScriptMoE lifts F1 on the CC-OCR end-to-end multilingual task from 65.71% to 80.89%, just above the best vision-language model's 80.73%, using a fraction of its parameter count

Why it matters

Multilingual scene text recognition has been stuck between two costly options: a dedicated recognizer per language, which inflates deployment cost and lets errors accumulate across a multi-language pipeline, or a large vision-language model, which is expensive to run and still misreads many scripts. ScriptMoE targets a single model that is simpler than the per-language route, lighter than a VLM, and more accurate than either, by letting a sparse mixture of script-aligned experts share one visual encoder.

Who it affects

The work is most relevant to teams building or operating OCR and document-processing pipelines that need to handle many languages and scripts at once, including existing systems like PP-OCRv5 where ScriptMoE was tested as a drop-in recognizer replacement, and researchers working on scene text recognition benchmarks and multilingual vision models.

How to use it

The paper demonstrates ScriptMoE as a recognizer module that can be swapped into an existing OCR pipeline, PP-OCRv5, in place of its default recognizer, with the rest of the pipeline left unchanged. No release date, licensing terms, or public availability of the TextMuSS-10M dataset, ScriptMoE model, or TextMuSS-Bench benchmark is given in the source.

How solid is it

The reported numbers, 82.06% accuracy on TextMuSS-Bench and an F1 lift from 65.71% to 80.89% on CC-OCR, come from experiments the authors ran themselves, including on a benchmark (TextMuSS-Bench) they assembled. The source gives no author names or affiliations and no publication date, so independent verification of the claims is not possible from this text alone.

Risks and caveats

The comparison against vision-language models is described only in relative terms, ScriptMoE uses a fraction of the parameter count, without absolute parameter figures for either ScriptMoE or the VLM it is compared to. The TextMuSS-Bench evaluation set was assembled by the same authors, and there is no mention of independent replication or peer review of these results.