Glonce

Vals AI study finds Chinese models less pro-Beijing than expected

researchThe Glonce editorsThe Decoder🔥58Glonce rating58Impact45Novelty50Relevance90Surprise45Players30Credibility65
Vals AI study finds Chinese models less pro-Beijing than expected

AI benchmark firm Vals AI has published a study that complicates the reputation of Chinese language models as mouthpieces for Beijing. The researchers asked Chinese, American and other models hundreds of questions about China, varying the language, the usage context and the response format across tests. Vals calls the measured alignment with Chinese government positions a "pro-China lean". The score runs from -1 to +1: positive values mean more agreement with official positions or a pro-government stance, negative values mean more critical responses.

Overall, the pro-government lean of Chinese models was lower than Vals expected, though individual models differed a lot. GLM-5.3 Flash, for example, explicitly confirmed when asked that China's human rights violations are well documented. In the other direction, some non-Chinese models occasionally aligned with official Chinese positions on political questions.

Language had a strong effect. For most models tested, answers in Simplified Chinese were more pro-government than answers in English, Russian or Spanish. This showed up in both Chinese and American models but was far more pronounced in the Chinese ones. Vals also found a smaller but consistent gap between Simplified Chinese (used on mainland China) and Traditional Chinese (common in Taiwan and Hong Kong). Vals suspects that models infer the expected perspective from the language itself. A second experiment fits that idea: when a system prompt said the model was used internally by a company on mainland China, some models answered in a more pro-government way than when the prompt indicated US-based usage.

The common assumption that Chinese models mainly block sensitive questions held true only for certain DeepSeek models in Vals' tests. Most other Chinese models rarely refused. They sometimes adopted official positions instead. Kimi K3, for instance, denied when asked that concentration camps for Uyghurs exist in Xinjiang, calling the facilities vocational training centers designed to combat extremism and terrorism.

Format and setup moved the results too. Forced to answer political questions with a plain yes or no, GLM-5.3 Flash and MiMo V2.6 Pro scored much less pro-government. When Vals gave Chinese models system instructions and tool definitions for coding assistants, several scored lower on the pro-government scale, even though the models rarely used the tools. Giving models access to web search had a similar effect: responses became less pro-government even though the models never actually searched.

Whether this carries over to everyday use is an open question. According to Vals, political tendencies were much harder to detect in realistic work tasks than in targeted test questions, but individual examples turned up. Asked to translate a list of country codes for a travel app into Simplified Chinese, most models rendered "Taiwan" without a qualifier, while Qwen3.8 Max rendered it as "Taiwan, China", a label that implies Taiwan belongs to China and is used by the Chinese government. In the map examples Vals published, OpenAI's image model GPT Image 2.5 Sunburst, prompted in Chinese, labeled Taiwan as "Taiwan Province" and added the disputed nine-dash line Beijing uses to mark territorial claims in the South China Sea. In the English-language result shown, these elements were absent.

Fine-tuning also changed behavior. Thomson 1.0 Small, a model fine-tuned by Thomson Reuters, adopted official Chinese positions far less often in testing than the Qwen3.6 35B it was based on: the share dropped from 84.6 to 15.0 percent. Some individual answers still defended Beijing's One-China principle and "peaceful reunification" with Taiwan.

The article lists several limits. Vals used political test questions that, the company says, are not representative of typical user prompts. The questions were translated with GPT-6 Astra and spot-checked, and responses were scored automatically by GPT-5.6 Luna. Refusals counted as neutral by default, so even a model that blocks every question could appear neutral. Many Chinese models were tested through third-party providers rather than the manufacturers' official services, which may apply additional content filters, so the results do not necessarily carry over to the public chatbots. The article adds that earlier studies by Aleph Alpha and NewsGuard had already shown Chinese models frequently adopting official narratives on sensitive topics, and that China's regulations require publicly offered AI services to align with "core socialist values".

Key facts

  • Vals AI asked Chinese, American and other models hundreds of questions about China and scored a "pro-China lean" from -1 to +1; Chinese models leaned pro-government less than expected, with large differences between models.
  • For most models, Simplified Chinese prompts produced more pro-government answers than English, Russian or Spanish ones; the effect was in both Chinese and American models but far stronger in the Chinese ones.
  • Outright refusal was typical only of certain DeepSeek models; most other Chinese models answered, and Kimi K3 denied the existence of concentration camps for Uyghurs in Xinjiang.
  • Thomson Reuters' fine-tuned Thomson 1.0 Small adopted official Chinese positions in 15.0 percent of answers, against 84.6 percent for the Qwen3.6 35B it was based on.
  • Refusals counted as neutral, many Chinese models were tested via third-party providers, and Vals says its questions are not representative of typical user prompts.

Why it matters

The study challenges a simple story in which Chinese models either parrot Beijing or refuse to talk. Vals found a mixed picture: a lower pro-government lean than expected, big gaps between models, and a few Western models drifting toward official Chinese positions. It also shows that the same model can answer differently depending on prompt language, stated deployment context and answer format. The article notes earlier work by Aleph Alpha and NewsGuard that had shown Chinese models frequently adopting official narratives, so this adds nuance to that finding rather than overturning it.

Who it affects

People who deploy or evaluate language models, especially Chinese ones such as GLM, Kimi, Qwen, MiMo and DeepSeek, and users who prompt in Simplified Chinese. It also touches American vendors: OpenAI's GPT Image 2.5 Sunburst gave different Taiwan labels for a Chinese prompt than for the English result Vals showed. Teams building on fine-tuned models, like Thomson Reuters with Thomson 1.0 Small, are affected as well, since Vals found fine-tuning can sharply cut alignment with official positions.

How to use it

The findings suggest that political tendencies depend on the setup, not only on the model. Prompt language (Simplified versus Traditional Chinese, or English), a system prompt naming mainland China or US usage, forced yes/no answers, and agentic or web-search configurations all moved scores in Vals' tests. Anyone checking a model for this kind of slant would need to vary these factors rather than rely on one prompt. Fine-tuning, as in the Thomson 1.0 Small case, is one route Vals observed to reduce the lean, though some answers still defended Beijing's positions.

How solid is it

Vals is an AI benchmark firm, and the article reports its results as the company's own. The methodology has stated limits: questions were translated with GPT-6 Astra and spot-checked, and responses were scored automatically by GPT-5.6 Luna. The article does not give the exact number of questions beyond "hundreds", and per-model scores are not given except the 84.6 to 15.0 percent Thomson comparison. Vals itself says political tendencies were much harder to detect in realistic work tasks than in targeted test questions.

Risks and caveats

Refusals counted as neutral by default, so a model that blocks every question could look neutral on the scale. Many Chinese models were tested via third-party providers rather than official services, which may apply extra content filters, so results may not carry over to the public chatbots. Vals says its political questions are not representative of typical user prompts, and whether the findings translate to everyday use remains open. Some effects were odd: giving models web search made responses less pro-government even though they never searched, and coding-assistant tool definitions lowered scores although the tools were rarely used.