Congress AI 'kill switch' plan has oversight gap

US lawmakers are considering legislation that would require AI companies to build the capability to immediately shut down their advanced AI programs. The bill, aimed at creating a "kill switch" mechanism, was introduced amid concerns about artificial intelligence potentially surpassing human control.

However, the legislation has a significant oversight: it does not account for open-weight (open-source) AI models. These models can be deployed locally, run on private hardware, and modified by users. Consequently, the proposed bill would be powerless against such models, even if they fall into the wrong hands.

"This isn't a hole in the fence. On the open-weight side, there is no fence," Rob T. Lee, chief of AI and chief of research at the SANS Institute, told Fast Company. He notes that modified models with stripped guardrails have been available for illicit purposes for some time, and local models are already sophisticated enough for many malicious activities.

The situation is further complicated by the growing interest from several US companies in Chinese open-weight models. These models offer competitive performance at significantly lower costs compared to major US programs. For instance, generating one million tokens can cost approximately $50 with US models, while corresponding Chinese models cost as little as $15 or even less than a dollar.

Experts point out that a complete ban on open-weight models in the US would be not only difficult to enforce but also ineffective. The models are essentially data files, and their distribution is virtually impossible to stop once they are on the internet. Companies aiming to restrict AI usage might inadvertently drive development underground rather than halting it altogether.