Enterprise AI KPIs collide with reality as scaling pressures mount
Executives across healthcare, energy, and consumer goods say long-term outcomes, not short-term KPIs, define success as AI scales enterprise-wide
As enterprise AI deployments mature, an increasing number of executives are questioning whether traditional performance metrics and Key Performance Indicators (KPIs) are fit for purpose. In highly regulated and complex industries, the value of AI systems often emerges years after deployment, challenging businesses under pressure to justify investment through near-term returns.
Rather than chasing immediate ROI, organizations scaling AI across operations are increasingly prioritizing predictive accuracy, scientific progress, and process transformation. The shift reflects a broader realization that enterprise-wide AI success depends less on model sophistication and more on aligning technology with long-horizon business and operational goals.
“In pharma, sometimes you don’t necessarily see the impact until years down the line,” said Danielle Belgrave, Vice President of AI and Machine Learning at GSK. “How do you quantify innovation? It’s important to see success in terms of whether we’re pushing the needle in science as well.”
She said one of the clearest technical indicators remains predictive accuracy, but added that this alone is insufficient when AI is applied to drug development. Identifying biomarkers that improve patient recruitment or reveal unexpected treatment responses can be decisive, even when outcomes remain uncertain for extended periods.
Executives in other sectors echoed the challenge of defining success in advance. In large energy systems and consumer-facing services, AI initiatives are often embedded within broader digital transformation programs. This makes it difficult to isolate the contribution of any single system.
“KPIs in AI are very good to do retrospectively. Doing them proactively is much harder,” said Claire Whitmore, Head of Digital Technology and Expert Services at E.ON UK. “It doesn’t matter which specific AI component moved the slider, as long as you are moving toward the strategy.”
The comments were made at Momentum AI 2025, a two-day enterprise technology conference organized by Reuters Events in London. The panel discussion, titled “Scaling Enterprise AI: Defining and Measuring Success,” was moderated by Christine Foster, General Manager for the GenAI Centre of Expertise at Experian UK & Ireland, and brought together senior technology leaders from healthcare, energy, and consumer goods.
Choosing what scales
Deciding which AI use cases deserve enterprise-wide rollout has become a central management challenge. At E.ON UK, that decision-making process is anchored in a structured value map aligned with corporate strategy.
“We have a value map, and every AI use case is assessed against it,” Whitmore said.
She said the framework evaluates whether a project improves customer experience, supports employees, optimizes the energy grid, balances supply and demand, or improves asset utilization. Most initiatives are expected to map clearly to at least one of those priorities.
“If a use case doesn’t map to that framework, it doesn’t mean we won’t do it, but we need a very good reason,” Whitmore said.
Bastien Parizot, Senior Vice President of IT & Digital at Reckitt, said prioritization began with baselining how employees spend their time. By identifying hundreds of discrete tasks and assessing which could realistically be augmented by AI, the company narrowed its focus to initiatives with measurable operational impact.
He said early results suggest that combining AI with human judgment is outperforming traditional approaches. This is particularly evident in product concept development, where AI-assisted concepts have shown significantly higher consumer resonance.
Parizot said the same pattern is emerging across other functions, including performance reviews and market analysis.
“We are not replacing human decision-making,” he said. “We are augmenting it by removing a lot of the groundwork and analytics, so people can focus on judgment and action.”
Process over models
As organizations move beyond experimentation, speakers emphasized that scaling AI depends far more on reworking internal processes than on selecting advanced models. At Reckitt, the focus has been on reshaping entire functions rather than deploying isolated tools.
“We follow a 70-20-10 approach. Ten percent of the change is technology, 20 percent is data, and 70 percent is about changing process and people,” Parizot said.
He said the most significant organizational shift has been converting tacit knowledge into standardized, reusable workflows that can be codified into AI systems. In areas such as marketing performance analysis or product development, teams previously relied on dozens of informal approaches, which made scale difficult.
“The biggest change AI brings is making tacit knowledge explicit and then codifying it into AI solutions,” Parizot said. “Probably the biggest change is how these tools are embedded into day-to-day ways of working.”
Belgrave said a similar discipline applies in pharmaceutical research. She said curiosity-driven exploration must be balanced against deployable impact.
“It comes down to having a product mindset with anything we do in the AI space,” Belgrave said.
She added that teams must be clear about whether an AI system is intended to generate scientific insight, improve operational efficiency, or be deployed at scale across the business.
Infrastructure choices
Foundational infrastructure decisions have also played a decisive role in determining whether AI systems can move beyond pilots. Speakers highlighted the importance of data reuse, cloud flexibility, and governance frameworks designed for scale rather than retrofitted later.
Parizot said Reckitt invested early in a shared data and analytics backbone. The goal was to ensure that cleaned and curated data could be reused across multiple applications.
“Enabling data reusability, large language models, and cloud optionality were the key foundations for us,” he said.
Parizot said the company deliberately avoided locking itself into a single large language model, as different business use cases require different capabilities.
Belgrave said that usability and iteration speed were critical when evaluating whether to build AI tools internally or procure them from external vendors. She said testing solutions quickly and refining them based on real-world use helped teams focus on systems capable of delivering sustained value.
Both executives said governance issues, including hallucination mitigation and observability, need to be addressed earlier than many organizations expect.
“Looking back, I would have invested in governance, hallucination mitigation, and observability even earlier,” Parizot said.
He added that these controls become significantly harder to implement once AI systems are already embedded in critical business processes.
Readiness and people
Speakers also challenged the idea that organizations must perfect their data foundations before deploying AI. Instead, they argued that well-chosen use cases can act as catalysts for improving data quality over time.
“Being AI-ready is being ready now, not waiting for perfect data,” Parizot said. “We actually improved our data foundations by running use cases, not before them.”
Whitmore drew a distinction between systems that work in isolation and those ready to scale. She said data pipelines and operational integration often become limiting factors only after early success.
Workforce impact has emerged as another unresolved issue. Parizot said Reckitt has begun tracking whether AI tools improve job satisfaction and learning. This is particularly relevant as knowledge is increasingly embedded in systems rather than acquired through experience.
Looking ahead, speakers said enterprises should resist the temptation to chase the latest models. Instead, they should focus on durability. Scaling AI requires technologies that align with existing strategies and processes, and metrics that recognize value even when results take time to materialize.