Anthropic bans users from being 'cruel' to its AI systems

AI company Anthropic has prompted debate by announcing it will ban users from "sustained and needless" abusive behaviour towards its AI systems. The company, known for its Claude AI model, updated its usage policy on Thursday to allow its tools to end interactions with individuals who are "cruel".

Anthropic clarified that the policy will only apply in "extreme cases" of repeated abuse. It will not cover "common versions of user frustration, pushback, dark creative themes, or model testing and research". Prohibited conduct also includes bullying others, promoting self-harm, and creating non-consensual intimate imagery.

However, what Anthropic defines as "abusive or cruel behaviour" towards its models remains unclear. Screenshots of the new policy have been widely shared on social media, sparking mixed reactions. Some praise the move as advancing "model welfare", while others criticize it, arguing that AI cannot be treated cruelly and that such anthropomorphism is harmful.

The changes, which also address risks like deceptive campaigns and election interference, have garnered more attention than previous policies on topics such as weapons development. The discussion reflects a broader debate about how humans should interact with AI tools and whether impolite communication could affect human interactions.