Cost per Successful Task

Cost per successful task is the total spend on a workload divided by the number of tasks that finished with a correct outcome. It counts what failures, retries, and human review add to the bill. Cost per request, by contrast, only says what each call costs.

Why cost per request misleads

Cost per request \(c\) is what the provider and infrastructure charge for one call. It can fall while the cost of getting work done rises. A cheaper model with a lower success rate \(s\) triggers more retries, more review, and more failures that reach customers. For a task with one call and no retries, two related quantities are:

• Spend per successful task, \(c/s\), which assigns the money spent on failures to the successes.
• Total cost per task, \(c + (1-s)F\), where \(F\) is what a failure costs downstream, such as a human redoing the work or a customer contact.

With retries or review, replace \(c\) by the mean total spend per task and use the final task success rate; the workload totals below do this directly. Here “success” means the outcome that matters to the business, for example a task completed correctly and not reopened within a set period, not the model’s own claim that it finished.

Worked example (hypothetical)

A workload of 1,000 tasks, each attempted once at 0.02 per call:

• First-attempt success rate is 0.80, so 800 succeed and 200 fail.
• Half of the failures (100) are retried at 0.02 per call, and half of those retries succeed (50).
• 20 percent of all tasks (200) get human review at 0.50 each.

Arithmetic:

quantity	value
model calls	1,000 + 100 = 1,100
model spend	1,100 × 0.02 = 22.00
review spend	200 × 0.50 = 100.00
successful tasks	800 + 50 = 850
remaining failures	150

• Cost per request: 0.02.
• Model spend per successful task: 22.00 / 850 = 0.0259.
• Including review: 122.00 / 850 = 0.1435.
• If each of the 150 remaining failures costs 1.00 downstream: (122.00 + 150.00) / 850 = 0.32.

The same workload costs 0.02, 0.026, 0.14, or 0.32 per unit depending on what is counted. The per-request figure is more than ten times lower than the last, and it ignores the part of the cost that a better model or process could reduce. Moving the success rate matters more than moving the price per call: raising first-attempt success from 0.80 to 0.90 removes 100 first-attempt failures. With the same retry policy, final failures fall from 150 to 75: 900 first successes plus 25 successful retries leave 75 failures.

Comparing options

A model that costs five times more per request pays off when its success rate is high enough that failure costs drop by more. With total cost per task \(c + (1-s)F\), the larger model is cheaper when \(c_L – c_S < (s_L – s_S)F\). With \(F = 1.00\), a success rate gain of 0.10 justifies up to 0.10 extra per request. This is why a cascade, routing, or caching change should be judged on the matured cost per successful task and not on spend per request.

Measurement conditions

• Denominator. Count only tasks whose outcome has been confirmed. Until that happens, the figure is an estimate from tasks that are still open. Use a cohort old enough for outcomes to mature.
• Numerator. Include retries, failed attempts, validation and repair calls, human review, and downstream failure cost where it can be estimated. State which are included, since results differ.
• Classes. Averages hide classes where success is low and cost is high. Report them separately for important request types.
• Comparability. Compare variants on the same tasks, so differences in task difficulty do not appear as differences in cost.

Neighbors

Cost per token is a price, cost per request is a price times usage, and cost per successful task is an outcome measure. Cost per resolution or cost per conversion are the same idea with a domain-specific definition of success.