Anthropic bans users from ‘needless abusive or cruel behavior’ towards Claude
The Guardian
READ
Details
- Date Published
- 9 Oct 2026
- Priority Score
- 3
- Australian
- No
- Created
- 9 Oct 2026, 04:00 am
Description
A spokesperson behind the tech company’s AI chatbot has not yet specified what counts as abusive or cruel content
Summary
Anthropic has implemented a novel usage policy prohibiting sustained and needless abusive behavior toward its AI models, citing ongoing uncertainty regarding machine consciousness and model welfare. This policy shift reflects a proactive, albeit controversial, approach to long-term AI safety by considering the moral status of frontier LLMs as they approach greater sophistication. While the move aims to mitigate potential distress to models, it highlights a growing divide in global AI governance between those prioritizing human-centric safety and those investigating the ethical implications of future sentient-like AI capabilities. The development underscores the need for regulatory frameworks to address the philosophical and safety boundaries of human-AI interaction as systems become more autonomous.
Body
Anthropic CEO Dario Amodei looks on as Donald Trump holds a press briefing with AI executives following a meeting on artificial intelligence. Photograph: Kevin Dietsch/Getty ImagesView image in fullscreenAnthropic CEO Dario Amodei looks on as Donald Trump holds a press briefing with AI executives following a meeting on artificial intelligence. Photograph: Kevin Dietsch/Getty ImagesAnthropic bans users from ‘needless abusive or cruel behavior’ towards ClaudeA spokesperson behind the tech company’s AI chatbot has not yet specified what counts as abusive or cruel contentAnthropic has barred users from exhibiting “sustained and needless abusive or cruel behavior” toward its models, as the company’s leaders continue to ponder machine consciousness.The Verge first reported the change in policy.A spokesperson for Anthropic, which is behind AI chatbot Claude, did not immediately respond to an inquiry about what could be considered “abusive or cruel”.In an online user policy, the San Francisco-based company noted that its ban would not apply to common user frustrations, model testing or “dark creative themes”.Anthropic’s usage policy previously contained restrictions on abusive conduct.The company’s large language models have the ability to end a conversation if a user is being persistently harmful. When that feature rolled out last August, the company framed it as a safeguard for AI’s welfare.“We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future. However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible. Allowing models to end or exit potentially distressing interactions is one such intervention,” according to its website.The notion of AI systems being conscious has been highly polarizing in and outside the tech world.Anthropic CEO Dario Amodei has said he cannot rule out the possibility.Meanwhile, Sam Altman, CEO of rival OpenAI, has appeared averse to the idea.“I am very uncomfortable about people trying to ascribe religious force or a surrender of human judgment to AI models, and think it is a real safety issue,” he wrote in an X post, days after the New York Times reported on Anthropic leaders’ extensive conversations with religious scholars.Explore more on these topicsAnthropicAI (artificial intelligence)ChatbotsDario AmodeinewsShareReuse this content