Enterprises buy AI infrastructure faster than they measure costs
A new study reveals that companies are acquiring AI infrastructure at a pace that outstrips their ability to track costs or measure utilization effectively.
Enterprises are accelerating their spending on AI infrastructure at a rate that significantly outpaces their capacity to monitor and control the associated economics, according to new research from VentureBeat.
The findings indicate that while most organizations currently rely on hyperscalers and model APIs for their AI operations, a substantial shift towards specialized compute is planned. This transition is happening despite a majority of companies being unable to clearly view their AI's unit economics. A key issue highlighted is the low utilization of existing GPU resources, with 83% reporting 50% or less utilization. Furthermore, fewer than half of enterprises (44%) can rigorously track their AI compute costs, creating a significant "compute gap."
This gap signifies that investment is outpacing visibility and control. Only about one in five (21%) businesses are running AI in production at scale, yet planned evaluations for AI-specialized clouds – a segment most are not currently using – are high. This suggests that purchasing decisions are being made with limited insight into the true cost and efficiency of current deployments.
Furthermore, vendor loyalty appears low, with a clear majority (64%) planning to switch or add an infrastructure provider within the next 12 months, and 38% intend to do so within the next quarter. When selecting providers, the primary drivers are integration with existing systems (41%) and total cost of ownership (35%), not headline prices like cost per million tokens (8%).
The research also points to a potential future bottleneck: the shift from GPU compute to memory bandwidth for scaled inference. However, awareness of this potential constraint is low, with only about one in five enterprises having addressed or being aware of it.