Enterprise AI agent evaluations show gap between autonomy and real-world performance
New research indicates half of organizations have deployed AI agents that passed internal tests but failed with customers. Only a small fraction fully trusts automated evaluation.
A significant gap exists in how enterprises evaluate AI agents, with a disconnect between automated testing and real-world performance, according to new research from VentureBeat. The study found that half of organizations have deployed AI agents that passed internal evaluations but subsequently failed when interacting with customers. Trust in automated evaluation is low, with only one in twenty expressing full confidence.
The primary reason cited for this lack of trust is that evaluations do not align with real-world outcomes. Despite this, companies are increasingly granting more autonomy to their AI agents. Alarmingly, two-thirds of organizations already permit, or are actively engineering toward, deploying agents to production based solely on automated evaluation, without human oversight.
The "Agentic Reliability & Evals tracker" research surveyed technical leaders to understand how AI agent performance is measured, what reliability and evaluation tools are used, and the extent of agent autonomy. The core finding is an "evaluation gap"—the disparity between the autonomy enterprises are handing over to their agents and their confidence in the tests designed to catch failures.
Findings reveal that while 50% of companies have experienced an agent causing customer-facing failures after passing evaluations, only 5% fully trust automated assessments. The most common criticism, at 29%, is the poor alignment of evaluations with actual usage outcomes. Yet, 66% of organizations are moving towards zero-human-in-the-loop deployments, signaling that increased autonomy is outpacing the development of robust assurance mechanisms.