AI inference routing recommendation: approve the scope that pays

Here is a concise recommendation for the supplied case. Assume the full annual savings and review costs begin immediately after implementation:

“I recommend routing the eligible routine requests to the cheaper model while keeping complex requests on the current model. That produces £75,000 in annual net savings after review costs and £35,000 in first-year cash benefit after the £40,000 implementation cost. Before rollout, I would confirm that the routing and review process preserves the required output quality, including for requests it classifies incorrectly.”

The table below calculates the annual and first-year cash effects from the opening figures. The net savings are derived results, not additional inputs:

If the interviewer asks for simple payback, £40,000 divided by £75,000 gives about 0.53 years, or 6.4 months of full operation. State the timing assumption. A rollout that takes months to reach full volume needs a separate cash schedule.

The route assignment is part of the system you are recommending. It cannot be assumed perfect just because the cheaper model performs well on a clean set of routine prompts.

Take a customer-support example. “Where is my parcel?” may be routine when it only requires reading a tracking status. It becomes more involved when the order contains multiple shipments, a disputed address or a refund request. The initial wording can look similar while the work differs.

Ask what happens when the routing rule sends such a request to the cheaper model. Can the system escalate? Does a reviewer catch the error? Does a failed first attempt add cost before the current model does the work anyway?

Test routing, review and fallback together on the work the company expects to receive. Good individual outputs do not establish that the wider application works reliably.

To see what could challenge the original forecast, consider a separate hypothetical test of 1,000 requests. The existing workflow costs £1,000 and produces 950 acceptable completed results, about £1.05 per accepted result. These test figures are an illustration, not observed results supporting the annual forecast.

The new workflow initially costs £600, but review and fallback add £450. It also produces 950 acceptable results. The total is £1,050, or about £1.11 per accepted result.

The cheaper initial model has made this workflow more expensive. A recommendation that quotes only the £400 reduction in the first model bill misses the decision.

The test does not automatically invalidate the supplied £75,000 annual forecast. It tells you exactly what must be reconciled: whether the forecast already includes the observed review and fallback burden at the proposed mix of work. Update the business case using that evidence before committing.

For the unit-economics setup behind this calculation, use the AI inference cost framework.

Return to the original annual forecast for this sensitivity: £100,000 gross saving and £40,000 setup cost. First-year cash is non-negative only if additional annual operating costs are no more than £60,000, assuming immediate full operation.

At the forecast £25,000 review cost, there is £35,000 of room before first-year cash reaches zero. At £70,000, annual savings would still be positive at £30,000, but first-year cash would be negative £10,000.

That is a better risk statement than “review costs may rise.” It tells the client when the risk changes the decision. If the client's requirement is payback within one year, £60,000 is the relevant operating-cost ceiling under these assumptions. If the client has a different return requirement, use it.

Include all extra recurring costs in that ceiling: review, routing infrastructure, fallback, monitoring and support. The label on the cost does not change the cash calculation.

The recommendation below remains conditional on validating the original forecast. If a real trial produced the higher costs in the illustration above, update the economics before approving rollout.

I would ask the product owner to define the routine categories and their acceptance requirements, then have the operating team measure the complete process before expanding. The finance owner should reconcile the result with the annual saving forecast.

Complex requests remain on the current model. A later proposal to move them needs its own quality and cost evidence because their failure modes and review effort may differ.

If the interviewer pushes, “Why leave savings on the table?”, respond directly:

“The routine segment already offers a positive return. We have not established the same economics for complex work. I would capture the supported saving and evaluate that segment separately.”

Give your recommendation with the original £25,000 review cost. Then have a partner change it to £70,000 and add a requirement for payback within a year.

Your decision should change: the revised proposal misses that requirement. Explain the £10,000 first-year shortfall and the operating cost or setup reduction needed to close it. Avoid keeping the same recommendation while merely adding a longer risk paragraph.

Fintech Support Cost Spike in the final-synthesis library gives you another setting in which a cost recommendation must survive the complete calculation.