AI token costs: worked out examples

The interviewer shares Exhibit 1: the two monthly bills and accepted-task counts. Accepted tasks meet the same completion and quality criteria in both months; the bill includes unsuccessful attempts and all charges within the same service boundary.

Calculate the cost per accepted task in each month. What would you ask for next to explain the change?

An API call is not necessarily a useful result. A task may involve several calls, a retry, a check and a fallback. Include the cost of failed work in the spending used to produce the accepted outputs.

Using the supplied figures, cost per accepted task moves from £300,000 ÷ 100,000 = £3.00 to £420,000 ÷ 110,000 = about £3.82, an increase of 27.3%.

The case confirms that acceptance means the same thing in both periods. Without that confirmation, ask: if last month required a checked answer and this month counts every generated answer, the two periods are not measuring the same outcome.

I would request the cost and accepted-task count by task type, alongside calls, input and output tokens, retries and separately billed services. That is enough to begin distinguishing mix from within-task performance.

In response to your request for a task-level split, the interviewer supplies Exhibit 2: routine and complex tasks for those same two months. Use it to test whether the higher average comes from a changed workload mix or higher cost within either group:

Previous spending is 80,000 × £2 + 20,000 × £7 = £300,000. Current spending is 70,000 × £2 + 40,000 × £7 = £420,000.

No task group became more expensive. The service completed many more complex tasks, which cost more even at unchanged efficiency.

At the old £3 average, the additional 10,000 tasks would explain £30,000 of growth. The remaining £90,000 comes from the changed mix in this decomposition. That gives the sponsor a much better explanation than “token efficiency deteriorated by 27%.”

A more expensive task may create more value. Suppose routine tasks generate £5 of contribution before AI cost and complex tasks generate £12. After the stated AI costs, each contributes £3 and £5 respectively.

Previous contribution after AI cost is 80,000 × £3 + 20,000 × £5 = £340,000. Current contribution is 70,000 × £3 + 40,000 × £5 = £410,000.

The AI bill grew by £120,000, but contribution after that bill grew by £70,000. On these fictional assumptions, the changed workload improved the business.

The client may still have a budget or capacity limit. That becomes a question about which useful work to fund, rather than an assumption that the larger invoice is inherently bad.

The supplied exhibit shows no within-group cost increase. In a different case, if complex tasks also became more expensive, break their workflow into the work actually billed. Look for longer documents, longer responses, extra tool calls, retries, different model routing or changes in repeated-input reuse.

A token count alone may not explain the invoice. Reconcile usage against the provider's actual billing categories and rates.

For an interview, you do not need to memorise a provider’s price table. Explain the accounting: quantity of each charged category multiplied by its rate, including unsuccessful attempts and other service charges.

Then choose a test that matches the driver. Excessive output length suggests testing shorter answers against the same acceptance requirement. Retries suggest investigating why the first attempt fails. A workload-mix change suggests a value and routing decision.

Suppose a separate trial processes 1,000 comparable tasks. The current route costs £7,000 and produces 950 accepted results. A cheaper route costs £5,000 initially, but review and retries add £2,600; it also produces 950 accepted results.

The current route costs about £7.37 per accepted result. The apparently cheaper route costs £8.00. Its lower initial price does not survive the full workflow.

If the cheaper route instead meets the same acceptance requirement at £6,000 all-in, it costs about £6.32 per accepted result and deserves consideration. The decision follows the measured outcome and total cost.

“Spend per accepted task rose 27.3%. In the task split we received, that is entirely a move towards more complex work, with unchanged cost within each group. I would check the value and budget allocation for that work before switching providers. Any model test should compare full cost per accepted result, including retries and review.”

Try Enterprise Sales Productivity Decline. The industry differs, but it tests the same discipline of choosing the driver analysis before the remedy. For the detailed cost equation, see AI inference costs.