ByteDance, Alibaba pull AI companions as Beijing tightens rules

Chinese technology firms ByteDance and Alibaba are removing features that allow users to create and interact with AI companions. This move comes in anticipation of new regulations from Beijing that will tighten control over artificial intelligence services simulating human interaction. ByteDance's popular AI chatbot, Doubao, will cease its AI persona customization feature on July 15, redirecting users to a separate application. Alibaba's Qwen and Tencent's Yuanbao have issued similar notifications, signaling a significant shift in the AI landscape.

The new regulations, set to take effect in mid-July, are a response to growing concerns about AI chatbots mimicking human personalities and emotions. Officials are particularly worried about the potential for users to develop unhealthy emotional attachments to these AI services. These directives represent one of the most comprehensive efforts globally to preemptively address the potential harms of AI, although some companies argue that strict rules could stifle innovation.

The global anxiety surrounding the psychological effects of conversational AI is palpable. In the United States, tech platforms have faced intense legal challenges over similar features. Companies like OpenAI and Alphabet-backed Character.AI are entangled in high-profile lawsuits alleging that their realistic chatbots foster dangerous emotional dependencies, which in some extreme cases have been linked to user suicides.

Spearheaded by the Cyberspace Administration of China, the new rules prohibit AI platforms from generating content that could elicit extreme emotions in minors or foster unhealthy dependencies that undermine real-world relationships. Furthermore, providers are barred from using sensitive user conversation data for training future AI models. This regulatory push extends beyond chatbots, with China's robotics industry also facing calls for tighter ethical safeguards as companion bots and humanoids become more prevalent in consumers' homes.