POLICY · MODELS · #2911
Vals AI study: Chinese models show varied ‘pro‑China’ lean; language and prompts shift responses
Vals AI tested dozens of Chinese, US and other language models on hundreds of politically charged questions and found that alignment with official Chinese positions varies widely by model, language, prompt framing and response format rather than uniformly reflecting a government ‘mouthpiece’. Examples include GLM‑5.3 Flash acknowledging documented human rights violations, Qwen3.8 Max rendering “Taiwan” as “Taiwan, China” in Simplified Chinese, and Thomson 1.0 Small (a Thomson Reuters fine‑tuning of Qwen3.6 35B) dropping pro‑government alignment from 84.6% to 15.0% in the tests; Vals also found non‑Chinese models sometimes adopted official Chinese positions and noted that refusals were scored as neutral and some tests used third‑party endpoints.
KEY POINTS
- Vals AI tested dozens of Chinese, US and other language models on hundreds of politically charged questions and found that alignment with official Chinese positions varies widely by model, language, prompt framing and response format rather than uniformly reflecting a government ‘mouthpiece’.
- Examples include GLM‑5.3 Flash acknowledging documented human rights violations, Qwen3.8 Max rendering “Taiwan” as “Taiwan, China” in Simplified Chinese, and Thomson 1.0 Small (a Thomson Reuters fine‑tuning of Qwen3.6 35B) dropping pro‑government alignment from 84.6% to 15.0% in the tests; Vals also found non‑Chinese models sometimes adopted official Chinese positions and noted that refusals were scored as neutral and some tests used third‑party endpoints.
- The findings matter because language, prompts and fine‑tuning can materially change political alignment in deployed models, affecting trust, moderation, geopolitical risk and how models are evaluated across jurisdictions.
WHY IT MATTERS
The findings matter because language, prompts and fine‑tuning can materially change political alignment in deployed models, affecting trust, moderation, geopolitical risk and how models are evaluated across jurisdictions.