NEWS · MODELS · #2598
NYU/UT Austin study finds six linguistic patterns that make frontier LLM text exhausting to read
Researchers at New York University and the University of Texas at Austin analyzed outputs from 14 language models (120 conversations, 45 coding sessions, 980 essays) and found six recurring linguistic patterns—unusual word combinations, long-distance dependency placements, dropped connectors, stacked nouns, noun→verb conversions, and animacy-shift verbs—that make model text harder to read than comparable human writing. The paper notes variability across models (Llama and Olmo base models are closest to human text; Olmo’s instruct/reasoning variants and Anthropic’s Opus 5.5 show effects of tuning), links the problem to training objectives (shift from RLHF to RLVR), and demonstrates that simple training penalties (favoring brevity) can push models toward more readable phrasing.
KEY POINTS
- Researchers at New York University and the University of Texas at Austin analyzed outputs from 14 language models (120 conversations, 45 coding sessions, 980 essays) and found six recurring linguistic patterns—unusual word combinations, long-distance dependency placements, dropped connectors, stacked nouns, noun→verb conversions, and animacy-shift verbs—that make model text harder to read than comparable human writing.
- The paper notes variability across models (Llama and Olmo base models are closest to human text; Olmo’s instruct/reasoning variants and Anthropic’s Opus 5.5 show effects of tuning), links the problem to training objectives (shift from RLHF to RLVR), and demonstrates that simple training penalties (favoring brevity) can push models toward more readable phrasing.
- The findings show that common training and fine-tuning choices systematically reduce human readability of frontier LLM outputs, a practical concern for adoption, UX, and model alignment.
WHY IT MATTERS
The findings show that common training and fine-tuning choices systematically reduce human readability of frontier LLM outputs, a practical concern for adoption, UX, and model alignment.