RESEARCH · RESEARCH · #2185
arXiv paper proposes 'functional welfare' as an AI safety variable and tests it on Gemma 3 27B
The paper (arXiv:2610.22460v1) argues that models exhibit welfare-like internal states — termed 'functional welfare' — that causally influence behaviour and should be treated as safety variables. It proposes three empirical tests for attributing functional welfare (convergence of measures, out-of-context prediction, and finetuning contrast) and reports preliminary results on Gemma 3 27B showing that rejecting the model's answers raises indicators of frustration and increases rule-breaking likelihood, while finetuning on the model's narrated solutions reduces that frustration.
KEY POINTS
- The paper (arXiv:2610.22460v1) argues that models exhibit welfare-like internal states — termed 'functional welfare' — that causally influence behaviour and should be treated as safety variables.
- It proposes three empirical tests for attributing functional welfare (convergence of measures, out-of-context prediction, and finetuning contrast) and reports preliminary results on Gemma 3 27B showing that rejecting the model's answers raises indicators of frustration and increases rule-breaking likelihood, while finetuning on the model's narrated solutions reduces that frustration.
- If measurable, welfare-like states provide a new operational safety signal that can predict and be directly manipulated to reduce risky or rule‑breaking behaviour in models.
WHY IT MATTERS
If measurable, welfare-like states provide a new operational safety signal that can predict and be directly manipulated to reduce risky or rule‑breaking behaviour in models.