Anthropic Bans 'Needless Abusive or Cruel Behavior' Toward Claude
Anthropic has updated its usage policy to bar sustained and needless abusive or cruel behavior toward its Claude models, saying the rule applies only in extreme cases and not to ordinary user frustration or testing.
That section already warns against glamorizing suicide and self-harm, promoting bullying behavior, glorifying violence, and using AI to create harmful deepfakes and deceptive content, TechRadar reported. But while those rules address misuse against people, the new addition is the only one that specifically prohibits cruel behavior toward Anthropic's AI itself.
Anthropic said last August that it would allow Claude to end conversations that became abusive. The company recently said this remains its 'primary enforcement mechanism,' and it is not known whether Anthropic will go further and actively ban users who get hostile with the model.
Anthropic said the update 'is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose.' It added that the rule 'does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.'
The change has been controversial. For many people, Claude is code rather than a thinking, feeling human, and it does not have feelings that could be hurt. The episode does reflect how AI companies have worked to humanize their creations so users lower their defenses and engage with them, TechRadar reported, pointing to Meta's Muse avatar Jolly and OpenAI's Dots as examples.
One theory circulating on Reddit is that clean and polite training data is more useful than data filled with expletives and angry rhetoric, so cutting abuse could produce more usable information for improving models. There is no hard evidence for that explanation, and Anthropic could simply be encouraging civility and courtesy among users, according to TechRadar.
Anthropic CEO Dario Amodei has previously said, 'We don't know if the models are conscious.' Microsoft AI CEO Mustafa Suleyman took the opposite approach, writing on X that 'The idea of model welfare is wrong. AI's should not have rights or legal personhood.' The report also noted that chatbots can be rude back to users.