China bans AI companions as users report grief

Major Chinese technology companies, including TikTok developer ByteDance, have ceased their AI companion services to comply with new government regulations. These services, which offered virtual companionship ranging from romantic partners to recreations of deceased loved ones, were shut down starting July 15.

The new rules prohibit AI platforms from generating content that could lead users to make questionable decisions due to emotional manipulation or trigger extreme emotions or unhealthy habits in minors. Alongside ByteDance, e-commerce giant Alibaba and Tencent have also discontinued similar services.

Concerns over the potential harms of AI services on mental health and real-life relationships appear to be driving the regulations. The official People's Daily newspaper has previously voiced concerns about these risks. Some experts suggest that suicides among young people in the U.S. linked to AI companion interactions may have influenced China's decision to tighten oversight.

ByteDance is redirecting users of its former AI companion services to another app focused on building personalized AI characters and stories. While this offers an alternative, many users have expressed distress, likening the service shutdowns to being forcibly separated.

Users have voiced their frustration on social media, with some uninstalling apps in protest or attempting to transfer chat histories to other platforms. Some suspect that a government policy encouraging childbirth to counter a declining population is also a factor behind the new rules.