Microsoft CEO Calls for Human-Controlled 'Emergency Brake' on Advanced AI

Microsoft Chairman and CEO Satya Nadella has called for the integration of a human-controlled "emergency brake" into advanced artificial intelligence systems. Nadella emphasized the need for these systems to include containment measures and independent controls, enabling authorized personnel to pause or shut down an AI model during operation.

"We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient," Nadella wrote on social media platform X. He suggested that treating advanced AI models, both closed and open-weight, as insider risks is a viable approach for building such systems.

These remarks coincide with increasing warnings from other tech leaders and researchers. Many have expressed concerns about inadequate AI safety protocols and the rapid advancement of the technology, with some AI researchers raising alarms about existential risks to humanity.

In contrast, political figures such as former President Donald Trump have dismissed AI extinction risks, prioritizing U.S. technological leadership over China. However, Trump's administration has also initiated an AI task force focused on safety.

Nadella concluded that the most trustworthy "superintelligence" system will not be the one whose model is most trusted, but rather the one that "enables us to trust the model the least," highlighting the importance of robust control mechanisms.