An AI bill has gone up. Someone suggests switching to a cheaper model. Before discussing models, establish what the business is paying to produce.
A retailer does not want a million cheap API calls. It wants accurate catalogue entries that can go live. If a lower-cost model needs more attempts and sends more work to editors, the apparent saving can disappear.
This worked example shows how to build a structure around that distinction, follow the numbers and recommend a test with a clear pass condition.
The case: cut the cost of accepted catalogue entries
An online retailer uses AI to turn supplier information into product descriptions. Its monthly model bill is rising. The operations director wants to reduce model cost per accepted catalogue entry by 20% within a quarter, while maintaining the current acceptance standard.
Human editors handle escalations. Their workload must also remain manageable.
All figures below are case assumptions. “Accepted” means an entry passes the retailer's existing checks for product facts, required fields and usable wording. Changing those checks would change what the cost metric means.
Your first questions should settle three things:
- Does the target cover model charges alone, or model charges plus human editing?
- Is the catalogue mix changing, for example towards products with longer or less complete supplier records?
- Are we comparing entries that meet the same acceptance standard?
The client confirms that the formal 20% target concerns model charges. You should still track editing cost and turnaround time alongside it. A recommendation that saves £1 in software and creates £3 of manual work is hard to defend.
Give the interviewer a structure they can follow
A useful opening sounds like this:
“I would first establish model spend per accepted entry and identify which product types account for the increase. Then I would separate the number of attempts from the cost of each attempt. That should tell us whether to focus on retries, the amount of information processed or the model used. Finally, I would test the strongest option against the same acceptance checks and include its effect on human editing.”
Each branch has a job:
| Question | What it would reveal |
|---|---|
| Which work is becoming more expensive? | Whether the change comes from volume, product mix or deterioration within a product type |
| How much model work is needed per accepted entry? | Retries, failed attempts, repeated context and unnecessary output |
| What do we pay for that work? | Model choice and the mix of billed input, output and cached content |
| What happens after the model responds? | Editing effort, acceptance and time to publication |
There is enough technical detail here to identify a cause without turning a business case into an architecture review.
Quick Structuring Drill
An insurer uses AI to draft claim summaries. Its cost per approved summary, including adjuster rework, has risen 30% in six months.
Sort each finding into the branch of cost per approved summary it affects.
Calculate the baseline before proposing a fix
For a comparable batch, the client supplies these figures:
| Measure | Current process |
|---|---|
| Accepted entries | 10,000 |
| Total model calls, including unsuccessful attempts | 12,000 |
| Average model charge per call | $0.10 |
| Entries escalated to an editor | 500 |
| Editing cost per escalation | $2.00 |
Model spend is 12,000 × $0.10 = $1,200. Dividing by 10,000 accepted entries gives $0.12 per accepted entry.
The target is $0.12 × 80% = $0.096 per accepted entry, or $960 for the same 10,000 outputs.
Notice the denominator. Dividing $1,200 by 12,000 calls would describe the price of a call. It would conceal the extra work required to get an acceptable result.
Also establish why the total bill rose. If accepted output doubled while cost per accepted entry stayed at $0.12, the higher bill reflects more production. There may still be a savings opportunity, but the diagnosis has changed.
Follow the cheaper model through the whole process
The interviewer now gives you a trial result. The alternative model costs an average of $0.06 per call, but needs 17,000 calls to produce the same 10,000 accepted entries. It also sends 1,200 entries to editors.
The model price is 40% lower. Model spend is not:
17,000 × $0.06 = $1,020, a saving of $180, or 15%.
It misses the 20% model-cost target. Including editing makes the result worse:
| Cost for 10,000 accepted entries | Current process | Cheaper model |
|---|---|---|
| Model charges | $1,200 | $1,020 |
| Editing | $1,000 | $2,400 |
| Combined cost | $2,200 | $3,420 |
The extra editing adds $1,400, outweighing the $180 model saving. Combined cost rises by $1,220.
I would reject a full switch on this evidence. I would then ask whether the alternative performs well on a specific part of the catalogue. Simple, complete supplier records may support a different approach from ambiguous records with missing specifications. The useful next step is a comparison by task type, rather than another average across everything.
Quick Case Math Drill
A telecom uses AI to draft customer email replies. A model charging half as much per call has finished a trial. Figures are per 1,000 replies sent.
- Trial model calls
- 2,000 at $0.02 each
- Agent rewrites
- 90 at $1.50 each
- Current combined cost
- $125
What is the trial model's combined cost per 1,000 sent replies, in $?
Investigate retries and repeated work
Retries deserve their own branch because they can reveal a problem upstream. Are required supplier fields missing? Does the prompt ask for an output format the downstream system rejects? Are successful results regenerated unnecessarily?
A retry caused by missing source information may remain a retry after changing models. An input validation step could address it more directly.
Repeated reference material is another possibility. Provider billing can distinguish ordinary input, output, cache creation and cache reads. The commercial question is how often the same material is reused and what that reuse costs under the chosen service.
Do not apply a headline caching discount to the whole bill. Output charges and uncached work remain, and creating a cache has its own billing treatment.
The structure should lead you to the relevant usage data. It does not require you to propose an implementation before seeing where the money goes.
Set a test that can earn a rollout
Use the same representative supplier records for both processes, including the difficult records that generate retries. Apply the same acceptance checks. Record accepted entries, total model charges, escalations, editing effort and turnaround time.
Define what a good output requires, then test it across a representative set of tasks. A handful of pleasing descriptions will not establish that the process meets the retailer's needs.
For this case, a proposal passes the model-cost hurdle at $960 or less per 10,000 accepted entries. It must also preserve acceptance quality and avoid an unacceptable increase in editing or publication time.
If one route improves simple products and harms complex ones, test a selective rollout. If neither meets the target, say which cost driver you would investigate next. A useful recommendation includes the evidence that would change it.
Practise the same logic in a different business
The transferable skill is choosing a useful output unit and accounting for the work required to achieve it. Delivery attempts per successful delivery follow a similar logic.
Try the delivery structure drill, then return to the profitability framework to connect unit economics with the overall profit change.
Structure drill
Would your case structure convince an MBB interviewer?
Last-Mile Delivery Cost Increase
Present your approach aloud. Get feedback on how you break down the problem, prioritise your analysis and explain your structure.
Frequently asked questions
Choosing the right cost measure
Why measure cost per accepted entry rather than cost per API call?
Because the retailer pays to get usable catalogue entries live, not to make calls. A cheaper call can still raise costs if more attempts fail. Dividing model spend by accepted entries counts retries and unsuccessful attempts in the unit the business actually values.
What counts as an 'accepted' entry in this case?
An entry is accepted when it passes the retailer's existing acceptance checks. This matters because the target is to cut cost per accepted entry while keeping the current standard. If a proposal quietly loosens the checks, any saving comes from a lower bar rather than a more efficient process.
How should I open the case so the interviewer can follow me?
Start with model spend per accepted entry and identify which product types drive the increase. Then separate the number of attempts from the cost of each attempt. That split shows whether to focus on retries, the amount of information processed or the model. Finally, test the strongest option against the acceptance standard.
What is the baseline cost per accepted entry?
Model spend is 12,000 calls at $0.10, or $1,200, which works out at $0.12 per accepted entry across 10,000 entries. Editing adds 500 escalations at $2.00, or $1,000. Keep both figures visible. The target concerns model cost, but editor workload is a constraint that any fix must respect.
Testing the cheaper model
Does a model that is 40% cheaper per call cut costs by 40%?
No. The alternative needs 17,000 calls instead of 12,000 to produce the same 10,000 accepted entries, so model spend falls only from $1,200 to $1,020. That is a 15% saving and misses the 20% target. The lower price per call is partly cancelled by the extra attempts.
What happens to the cheaper model's case once editing costs are included?
It becomes more expensive overall. It sends 1,200 entries to editors, which at $2.00 each costs $2,400 against $1,000 today. Added to $1,020 of model spend, the process costs $3,420 against $2,200 currently, and editors would face more than twice the escalations.
What model spend would hit the 20% reduction target?
Model spend per accepted entry would need to fall from $0.12 to about $0.096, or roughly $960 for the 10,000-entry batch. Stating this early lets you judge each option quickly. The cheaper model's $1,020 falls short before editing costs are even considered.
If this cheaper model fails, does that rule out switching models altogether?
Not necessarily. The trial shows that this particular model fails on these figures, not that every alternative will. A switch might still be worth testing after retries are reduced. The rule is that any option must be judged on accepted output, editor workload and the acceptance standard together.
Fixing retries and proving the result
Why do retries deserve their own branch of the structure?
Retries can reveal upstream problems that a new model will not fix. Examples include missing supplier fields, an output format the downstream system rejects, or successful results being regenerated unnecessarily. If the cause is missing source information, an input validation step may address it more directly than changing models.
Could repeated reference material be inflating the bill?
Yes, it is worth checking. If the same reference content is sent with many requests, the amount of information processed rises without improving output. Ask how the provider bills that repeated material before blaming the model, because the remedy may lie in how requests are put together.
How should I design a trial that could justify a rollout?
Run both processes on the same representative supplier records, including the difficult ones that trigger retries, and apply the same acceptance checks. Record accepted entries, total model charges, escalations, editing effort and turnaround time. A handful of pleasing descriptions will not show that the process meets the retailer's needs.
Where else can I practise this kind of reasoning?
Practise on any case where the useful output differs from the activity being paid for. Delivery attempts per successful delivery follow the same logic: count all the work needed for one good outcome. Then connect those unit economics back to the overall profit change using a profitability framework.