China's Ministry of State Security Warns of AI Voice Cloning Risks

China's Ministry of State Security has issued a warning regarding the potential risks associated with AI-powered voice cloning technology. The ministry highlighted concerns that as AI becomes capable of rapidly and accurately replicating human voices, its misuse could disrupt social order and endanger public property.

The ministry identified three key characteristics of AI voice cloning: its speed, enabling voice cloning in as little as 3 to 5 seconds; its realism, with studies showing the public struggles to distinguish high-quality synthesized voices from real ones; and its ease of use, with many online platforms offering "voice cloning" features to general users.

These capabilities present three primary risks. Malicious actors can collect voice samples from victims' families to synthesize fraudulent calls, creating urgent scenarios for financial scams. False audio and video content using cloned celebrity voices have appeared on various platforms for advertising and traffic generation, infringing on individuals' rights. Furthermore, cloned voices of public officials and figures can be used to fabricate false statements or policy interpretations, spreading misinformation through social media.

To counter these threats, China has established a legal framework. The protection of an individual's voice is treated similarly to portrait rights under the Civil Code, making unauthorized cloning, synthesis, or distribution of recognizable speech an infringement. Regulations such as the "Interim Measures for the Management of Generative Artificial Intelligence Services" mandate platform responsibilities, including user verification and content traceability.

The Ministry of State Security urges online platforms to uphold their primary responsibilities by removing illicit tools, enhancing content review, and implementing warning mechanisms. Industry participants are called upon to adhere to ethical standards, while the public is advised to safeguard personal voice data, use multi-factor authentication for sensitive verifications, and report instances of infringement or fraud to authorities.