AI Risk Management Lacking as Firms Deploy Agentic AI, Expert Warns

Companies are accelerating the deployment of agentic artificial intelligence without fully understanding how to manage the associated risks, according to Steven Mills, a partner and managing director at Boston Consulting Group (BCG).

Mills, who also serves as BCG's Chief AI Ethics Officer, expressed concern that businesses are moving too quickly into agentic AI without sufficient controls, a move that could lead to serious repercussions. "The desire to move fast on AI without putting appropriate risk management in place can result in a system lapse or use cases being deployed in areas with strong regulatory requirements that organizations are unprepared for," he stated in a blog post.

Facing immense pressure to demonstrate AI-driven productivity gains, many business leaders have told Mills that their risk management programs are "cumbersome and slow" and unable to keep pace with AI's exponential growth. Mills emphasized the high stakes, noting, "Get governance wrong, and every bit of value you’ve built with experimentation and early wins could unravel because of a single incident."

The warning comes as other AI professionals raise similar alarms. Gartner previously predicted that 40% of enterprises will have to deactivate autonomous AI agents next year after governance gaps are exposed by "production incidents." The consequences of AI misuse are not new; in 2023, the Federal Trade Commission banned Rite Aid from using facial recognition technology for five years after its AI system falsely identified thousands of customers as suspected shoplifters.

Mills advised companies to establish systems that differentiate between low-risk use cases and those requiring deeper review, with higher-risk AI products necessitating human oversight. He also recommended allocating sufficient budgets for governance and appointing a senior executive accountable for AI safety. "The key is strategically injecting the stage gates and reviews needed to manage AI risk—and making it a priority. Outcomes of the system should deliver value versus doing harm."