Success Criteria for AI Consulting: How to Measure ROI

How to measure ROI of AI consulting starts before the work does. In IBM's 2025 CEO Study of 2,000 CEOs, only 25% of AI initiatives have delivered expected ROI over the last few years. Yet 68% of the same CEOs said their organization has clear metrics to measure innovation ROI. Metrics at company level are one thing. Criteria agreed for one piece of work are another.

The short answer: before an AI-assisted research or consulting engagement starts, agree four things. They are the decision it serves and its deadline, the evidence standard, a baseline, and what your team keeps and can rerun. Check the same four afterwards. Faster delivery counts only if the method stays with you.

Are you interested in more? Book time with us!

Why "faster and cheaper" is not a success criterion

Faster and cheaper describe how the work was delivered. They say nothing about whether it worked. An engagement can finish in half the time and still fail if no decision moves, or if the next edition starts from a blank page. With AI, the speed is easy to claim: a report that took weeks can arrive in an afternoon.

Speed turns into value in two ways only. The time saved goes into more or better work, or the method that produced the result stays with your team so the next round costs less. If neither happens, you paid for a faster one-off. The four criteria below test both.

1. The decision it serves, and its deadline

Every engagement should name one decision, the person who owns it and the date it has to be made. "Understand the hydrogen market" is a topic. "Decide by 30 November whether to fund a pilot plant in two regions" is a decision.

Write three lines into the brief:

• The decision: what will be chosen between, in one sentence.
• The owner: the person who signs it off.
• The date: when the decision is due, and when the work must land to be of use.

Work that serves no decision fails, however good it is. This is also the easiest criterion to check afterwards. The decision was made on time with this work on the table, or it wasn't.

2. The evidence standard

The evidence standard says what every claim must carry before it reaches the decision maker: a source you can open, a check someone ran, and a label when it isn't verified. With AI in the loop, this matters more. In a 2025 study by 22 public service broadcasters, 45% of AI assistants' answers had at least one significant issue, and 31% had significant sourcing issues (EBU/BBC report).

Agree in writing:

• Sourced and checkable. Every figure and claim links to the page it came from.
• Sources collected apart from the conclusions. Gather the evidence first, then argue from it, so a reviewer can see what was found before anyone decided what it meant.
• Named quality checks. Say who or what checks the output, and against what.
• Labels for the unverified. Projections, estimates and single-source claims are marked as such and kept apart from verified facts.

What AI can't do here is carry the judgement. Someone still has to verify each claim and citation the tools give them. An evidence standard builds that check into the work, instead of leaving it to whoever reads the report last.

3. A baseline measured before the work

A baseline is the same measure taken on the old way of working, before the engagement starts, so you have something to compare with afterwards. The NIST AI Risk Management Framework makes the point for any AI meant to augment or replace human activity, including decision making: it "requires some form of baseline metrics for comparison."

Pick one named task that repeats, such as a monthly market report, a quarterly competitor review or a screening memo. Record these measures on the current process:

Take the same measures on the same task afterwards. Without a baseline, any claim of time saved is a guess. Hours saved is a useful number, but alone it isn't a success measure: it counts only if the time goes somewhere.

4. What the client keeps and can rerun

The fourth criterion is what stays with your team when the engagement ends: the method written down, the evidence preserved, and people able to run it again without the provider. It's the difference between buying an answer and building decision capability instead of renting confidence.

Agree up front:

• The method, written down where your team can read and change it.
• The evidence, preserved: the sources and the facts drawn from them, not only the final slides.
• A rerun by your own team on the next real edition, booked before the engagement closes.
• Ownership of outputs and data, stated in the contract.

A boutique global brand strategy consultancy put this to the test on its own method. FifthRow encoded the firm's discovery method as a running system. It returned the lead territory the firm's strategists had already built by hand, in 48 minutes and without guidance. The framework has since been reused across ten engagements and pitches instead of being rebuilt each time (how the consultancy's discovery method was encoded). Once a method runs as a system, it can also run on a schedule, as with Autopilots for recurring reports.

Whether to build that capability in-house or buy it is a separate choice. Our post on the build-vs-buy choice for multi-agent platforms covers it.

How to tell afterwards whether it worked

Check the same four criteria on one page, a few weeks after delivery, with the decision owner in the room.

The ROI figure then comes from numbers you already hold. Take the baseline cost per study, subtract the new cost per study, multiply by the editions a year, and set it against the fee. The value of a better decision is harder to price. Record what was decided and what happened next, rather than putting a made-up number on it.

If one row fails, you know where to look. A missed date points to criterion 1, an untraceable figure to criterion 2, and a rerun that never happened to criterion 4.

Where FifthRow's view fits

FifthRow is The Agentic Consultancy: we deliver consulting work to decision-ready quality, then leave the client owning the capability that produced it. We deliver the outcomes, or your team runs the work on FifthRow. The four criteria above are the ones we'd want to be judged on.

A $1B+ regional German energy utility publishes an internal report every month on the regulation, technology and market shifts moving its sector. Assembling it by hand took around ten hours an edition and capped how much evidence could go into it. On FifthRow, an edition takes roughly 90 minutes end to end and rests on 247 answered questions, each carried on verified primary material. Because the person who writes the report configured the first version, the reasoning behind it is documented where colleagues can read and change it (how the utility's monthly report runs).

That is criteria 2, 3 and 4 in one case: sourced evidence, a measured before-state, and a method the team owns. For agentic market intelligence, the same principle means prior work stays available, so the next cycle does not start from zero.

Are you interested in more? Book time with us!

Frequently asked questions

How do you measure the ROI of an AI consulting engagement?

Measure a baseline on one recurring task before the work starts: hours, elapsed time and cost per study. Take the same measures afterwards, multiply the difference by the editions a year, and set it against the fee. Then add two checks a number can't capture: was the decision made on time, and can your team rerun the work.

What should be agreed before an AI research project starts?

Four things: the decision it serves, with its owner and deadline; the evidence standard; the baseline; and what your team keeps and can rerun. Put them in the brief or the contract, so both sides check against the same list when the work ends.

How do you check the quality of AI-generated research?

Pick claims at random and open their sources. Each one should lead to a page that says what the report says. Check that projections and single-source claims are labelled, and ask which quality checks ran on the output and who reviewed it.

What should a client own when the engagement ends?

The written method, the preserved evidence, the outputs and the data, and the ability to run the work again. The test is a rerun by your own team on the next real edition, booked before the engagement closes.

Is hours saved a good measure of success?

Only as one part of it. Hours saved count when they go into more or better work, or when the next edition costs less because the method stays with you. Without a baseline measured beforehand, an hours-saved figure is an estimate.

Agree the test before the work starts

The ROI of AI consulting is decided at the start, when the decision, the evidence standard, the baseline and the rerun are written down. Check the same four afterwards. If your team can run the next edition without the provider, the speed paid off.