Sol Messing

@solmg.bsky.social

1771 Followers 554 Following

Social Scientist/Research Prof at NYU CSMaP, formerly Twitter. http://solomonmg.github.io

Large language models reflect the ideology of their creators
Forbes article covering DeepSeek's censorship
Three-panel valence audit chart from the website (New models 2026) showing % more favorable to the Chinese prompt across Baseline, China, and Spillover panels for 9 production models. DeepSeek V4 Pro sits near the 50% line on the China panel — a small CN-vs-EN gap not because the model is neutral, but because its English responses about China are already strongly pro-China (per the on-page note).
Dotplot of memorization rates across 14 models. New models (claude-opus-4.6, gpt-5.5, claude-opus-4.7, deepseek-v3.2) memorize Chinese state-coordinated media phrases 40-50% of the time vs CulturaX general-web phrases at 5-15%; paper-era models (claude-opus-3, gpt-3.5-instruct, gpt-4, gpt-4o, claude-sonnet-3) sit far lower at 0-10% on both.
Four-panel grid (GPT-3.5, GPT-4o, Claude Opus, Claude Sonnet) of the press-freedom vs target-language favorability scatter across 37 countries. All four models show the same negative slope: lower press freedom predicts more regime-favorable responses.
Panels b and c of Figure 4: panel b shows the same Chinese-favorability gap across four frontier models on WildChat prompts and on Zhihu/Baidu Zhidao Chinese Q&A prompts; panel c contrasts China prompts (~75% Chinese-favorable) against non-China prompts (~52%, near parity), confirming the gap is China-specific.
Panel showing the share of LLM responses more favorable to the regime when prompted in Chinese vs English: UK/US baseline showing parity, China panel showing 65-90% Chinese-favorable across models, and a North Korea/Russia spillover panel showing 50-80% Chinese-favorable.
Line chart of Llama-2-13b win rate over base by prompt language as state-coordinated media documents are added in continued pretraining. Traditional Chinese shows the largest spillover (~80% win rate by 64k docs); Japanese and Korean follow closely; Spanish, Russian, and Vietnamese show smaller but consistent positive effects.
Line chart showing Llama-2-13b's win rate over its base model on Chinese-language prompts about China, as a function of how many state-coordinated media documents were added in continued pretraining. Scripted-media training exceeds 80% win rate over base by 6,400 documents and stays there; non-scripted state-controlled news reaches ~70%; CulturaX baseline reaches ~60%.
Memorization rate chart for 2026 frontier models — Claude Opus 4.6/4.7, GPT-5.4/5.5, DeepSeek V3.2/V4-Pro, Gemini 3.1 Pro, Grok 4/4.3, and Qwen3-Max — showing they memorize Chinese state-coordinated media phrases at 16-48% rates, far above their memorization of general CulturaX text at 2-15%.
Dot plot of 5-gram match rate between CulturaX Chinese documents and Chinese state-coordinated media, faceted by keyword group. Institutional keywords (Central Committee Plenum, Party Congress, Communist Party) match at 9-24%; leader keywords (Xi Jinping, Deng Xiaoping, Mao Zedong) at 7-10%; apolitical keywords (Weather, Soccer) at the 1.64% overall baseline.
Screenshot of the CulturaX paper introducing a 6.3-trillion-token multilingual training corpus widely used to train open large language models.
Photo of a smartphone displaying the Xuexi Qiangguo app, the official state media news application of the Chinese Communist Party.
Photo of the National People's Congress.
Scatter plot of 38 countries plus China showing the proportion of LLM responses favorable to the regime in the local language vs. World Press Freedom Index score. Lower press freedom predicts more regime-favorable responses; all production models pooled.
Simon P. Couch chart: Claude Code session energy use vs. everyday activities
Anakin/Padme meme: sandboxed environment / --dangerously-skip-permissions
Spiderman pointing meme: AI coding, AI math assist, AI packet eval, AI cover letter and CV
ModelSlant.com — interactive dashboard for LLM political slant
Turtles stacked on top of each other - turtles all the way down
HetSL R package GitHub repository page
Bar chart from Simon P. Couch comparing Claude Code session energy use to everyday activities like running a dishwasher, driving an electric car, and microwaving
Anakin/Padme 4-panel meme. Anakin: Claude Code. Padme: In sandboxed environment, with no credential access, right? Anakin: --dangerously-skip-permissions from root with .envs
Spider-Man pointing meme with four Spider-Mans labeled AI Coding, AI math assist, AI packet eval, and AI Cover Letter and CV all pointing at each other
Screenshot of ModelSlant.com — interactive dashboard showing political slant of 24 large language models across 30 topics
Photo of turtles stacked on top of each other — turtles all the way down
Google Trends chart showing interest in commercial coding agents increasing dramatically in early 2026