top of page

AI Doesn't Need a Picnic. It Needs a Business Problem.

3 minutes ago
4 min read

Every CFO has asked some version of the same question this year: we spent money on AI, so where's the return? Most companies answer with adoption numbers, license counts, or a survey where employees estimate how many hours AI saved them last week. None of that holds up well in a budget meeting. A growing number of executives running real AI programs are saying so out loud.

Kartik Sakthivel, CIO at John Hancock, put it about as directly as anyone could at a recent industry conference on AI ROI. "AI cannot be a sandwich looking for a picnic," he said. Meaning: don't start with the technology and go hunting for a use case. Start with the business problem, and let AI show up only if it actually solves it.

That sounds obvious. In practice, it isn't. Sakthivel inherited a public commitment from John Hancock's parent company, Manulife: a billion Canadian dollars in AI-driven value by the end of 2027. Most CIOs would run from a number like that. He says it's the opposite — a hard number forces clarity, because you can't hand-wave your way to a billion dollars with vague productivity claims.

His clearest example is Quick Quote, an internal tool that gives customers and their agents an early, non-binding read on life insurance eligibility. Before, that first pass took a day or more. Now it takes about 15 minutes. That's not a productivity anecdote. It's a number a CFO can put in a spreadsheet: more applications moving per underwriter, faster time to a policy, and underwriters spending their time on the complex cases that actually need human judgment instead of the routine ones a model can screen.

What Sakthivel won't do is treat self-reported time savings the same way he treats that kind of hard number. He calls it "the trust me, bro metric" — his own name for the surveys where employees guess how many hours AI saved them. He's not dismissing it outright, just honest that it's shaped by incentives most leaders don't think about. His own example: if he's actually saving twenty hours a week, he's more likely to report five, because he doesn't want a manager loading him up with twenty hours of new work in return. Getting an honest number out of that kind of metric is a leadership and trust problem, not a measurement problem.

Elsevier's CTO, Jill Luber, backed up the same instinct from a completely different industry. Elsevier posted 6% revenue growth in the first half of 2026, its strongest growth in years, and she ties a real share of it to specific, narrow AI investments rather than a general AI strategy. One example is a tool her team built called a claims radar: it takes a scientific claim inside a research article and searches the rest of the literature for everything that supports it, contradicts it, or complicates it. Researchers use it on other people's work, and increasingly on their own, before it goes to peer review, so they can see where the pushback is coming before a reviewer finds it for them.

One detail stuck with me. Elsevier built knowledge graphs and taxonomies on top of its own content library once, up front, rather than making every user query re-derive the same relationships from scratch. Luber's team absorbs that token cost so their customers don't pay it every time they ask a question. It's a tradeoff that only shows up when you're tracking cost per outcome instead of counting logins.

Luber was just as blunt as Sakthivel about which metrics don't count. "Adoption's not one," she said — meaning people logging into a tool tells you nothing about whether the tool changed an outcome. Her team still tracks the DORA metrics software teams have used for years: release velocity, bug rates, time to production. AI hasn't retired those. It's added a layer on top, including how many versions of something a team can build and throw away before landing on the one that ships, since fast, cheap iteration is new territory AI has actually opened up.

A third executive at the same event, Anthony Caiafa, Global CTO and CIO at SS&C Technologies, put a finer point on the same problem from the engineering side. Pull request volume, a metric a lot of engineering leaders lean on to show AI coding tools are paying off, turned out to be a poor one in his experience. Teams learned to game it fast, padding the count with trivial one-line changes that look like productivity and cost real token spend to produce.

Put those three together and a pattern falls out. The companies actually showing results aren't measuring whether people are using AI. They're measuring whether specific, narrow AI investments moved a number that already mattered before AI existed: underwriting throughput, revenue, release velocity, customer satisfaction. Adoption, token counts, and self-reported hours saved are easy to collect and easy to report up the chain. They're just not proof of anything.

If your AI ROI story right now is a usage dashboard, it's worth asking what number you'd show a skeptical CFO instead. The companies getting real credit for AI this year already know the answer. They measured the business, not the tool.

 
 
 

Comments


© 2025 by Tom Smith

bottom of page