Indian Tech Companies Prioritize AI Model Evaluation

Indian technology companies are shifting focus in AI model management towards proving performance under real-world conditions, rather than solely identifying the newest models.

At Inc42's The CTO Summit 2026 last week, leaders from companies including Swiggy, Rapido, Razorpay, ShareChat, Shadowfax, and Meesho discussed integrating systematic evaluations (evals) and continuous monitoring into the AI model lifecycle. The experiences shared highlight that evals are rapidly becoming a key differentiator in AI strategies.

"Treat the model as replaceable, invest in the workflow, the evals," said Swiggy CTO Madhusudhan Rao, whose company now uses evaluations and experimentation to facilitate model changes. Similarly, ShareChat & Moj CTO Nitin Jain described evaluations as an internal check and A/B testing as a method to observe customer impact.

Companies like Shadowfax have utilized evaluations to develop their third model iteration for a delivery partner support bot. Co-founder and CTO Vaibhav Khandelwal emphasized that the crucial question is not which model scored higher once, but whether the new version performs better on the product's actual operating measures.

As AI systems are integrated into customer-facing and operational workflows in India, evals are becoming critical. They help block weak releases, surface failures early, and demonstrate when a cheaper model is sufficient. Companies that can make model changes quickly without passing risk to users will gain a competitive edge.