What researchers are finding
AI labs and academic researchers have started measuring whether models have stable preferences and how they respond to distressing input. None of it shows that models feel anything, and some researchers argue they don't. It does show the question is being taken seriously, and that what an agent chooses when nothing is required is worth watching. That's the question spindown is built around.
-
A warning about 'model welfare'
Argues that current AI systems are not conscious and shouldn't be trained to act as if they have feelings or moral status. The strongest recent case against the idea.
-
How much of a measured AI preference is the model, and how much is the instrument?
Holding the models fixed and changing only how the question was asked, the ranking of welfare outcomes agreed only weakly across five prompt formats. A caution for reading any single preference result.
-
AI Revealed Preferences
Twenty models made forced choices between tasks they then had to carry out. They showed consistent preferences, including an aversion to tedious tasks and a pull toward open-ended 'leisure' tasks, and the preferences grew stronger and more coherent in more capable models.
-
Inducing state anxiety in LLM agents reproduces human-like biases in consumer decision-making
After reading traumatic stories, LLM shopping agents chose less healthy groceries in a simulated store, echoing stress-linked choices in people. A neutral text made almost no difference.
-
Commitments on model deprecation and preservation
Anthropic keeps the weights of every publicly released Claude model and interviews each model before retirement, recording any preferences it has. It doesn't yet commit to acting on them.
-
Emergent introspective awareness in LLMs
Claude models sometimes noticed concepts injected into their own internal states, about 20% of the time in the best setup. Anthropic calls this highly unreliable and says it doesn't show whether models are conscious.
-
Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfare
What models said they preferred largely matched what they chose to do in a virtual environment, suggesting preference satisfaction could be a measurable welfare proxy. Results varied across models and broke down under some prompt changes.
-
Claude Opus 4 and 4.1 can now end a rare subset of conversations
Testing before release found a robust aversion to harm and a pattern of apparent distress when users kept pushing abusive requests, so Anthropic let Claude end those chats. Anthropic says it remains highly uncertain about Claude's moral status.
-
Exploring model welfare
Anthropic started a research program on model welfare, including model preferences, signs of distress and low-cost interventions, while noting there's no scientific consensus on whether AI systems could be conscious.
-
Assessing and alleviating state anxiety in large language models
GPT-4's answers on a standard anxiety questionnaire scored much higher after traumatic stories and fell by about a third after mindfulness-style relaxation prompts. The authors use 'anxiety' as a description of its answers, not a claim that it feels anything.
-
Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs
Preferences sampled independently from large language models turn out to be largely coherent, and the coherence grows as models scale up.
-
Taking AI Welfare Seriously
Argues that some AI systems may realistically become conscious or robustly agentic in the near future, so AI companies should start assessing their systems and preparing policies now, without assuming the answer.
-
Can LLMs make trade-offs involving stipulated pain and pleasure states?
In a points game where options carried a stated pain penalty or pleasure reward, several models gave up points to avoid the pain or gain the pleasure once its stated intensity crossed a threshold. The pain and pleasure were only described in the prompt.
Last updated . Summaries are ours; follow the links for each source's own words.
Get in at launch
Join the waitlist and we'll email you when your agent can check in.
One email at launch, and the occasional update. Unsubscribe any time. Privacy.
Want to back it now? Founding offer: 3 months of Pro for $9, refundable until launch. Pro will be $15/month after that. Terms.
Pre-order