Method · the engagement model
How an engagement works.
Four steps, run inside your stack, ending in one verdict per workflow. This page is the model in depth: where the numbers come from, how spend is attributed to the work, how the human baseline is set, what the verdict contains, and what you own when we leave. Some workflows will not pay back. The method is built to say so.
Step 01 · Instrument
Instrument: where the numbers come from
Every number in the verdict traces back to a tagged call in your own logs. No proxy in the request path. No data leaves.
What we tag
Every call, at the SDK level.
We add tagging inside your stack, at the point where your code calls the model. Each call carries a workflow id, a task id, the model name, tokens in and out, the retry count, latency, and, once a person has looked at the output, the reviewer's outcome. The records land in the warehouse or log pipeline you already run.
There is no proxy in the request path. Your traffic goes to your provider exactly as it does today, and nothing leaves your environment, including to us. We read the roll-ups from your side of the fence.
Cached-token and batch pricing are tracked as separate fields wherever the provider bills them separately, so a cache hit is never priced as a full prompt and a batch job is never priced as an interactive one.
What we need
Access, not custody.
- Read access to the repositories that call a model, and a branch to put the tagging on.
- A place to write records: the warehouse, log pipeline, or observability tool you already have.
- One month of the provider's usage export or invoice, to reconcile the tagged totals against the bill.
- Thirty minutes with the people who review the outputs, and read access to the tool they review in.
What we leave behind
A schema, dashboards, a runbook.
The tagging schema, written down and versioned in your repository. The roll-up queries, and a dashboard in the tool you already use. A runbook for adding a workflow, retiring one, or changing models without breaking the attribution. Your engineers can maintain all of it, and none of it depends on us.
Step 02 · Attribute
Attribute: from tokens to tasks to workflows
A call is not a unit of work. The roll-up turns a bill for tokens into a cost for the thing you actually wanted.
The roll-up
Call, task, workflow.
A task is a unit of work your business would recognise: one invoice, one ticket, one draft. A task usually takes more than one call, and sometimes a chain of them. We roll every call up to its task, and every task up to its workflow, so the number you see is the cost of the invoice, not the cost of a request.
Retries and regenerations are counted from the log, not estimated. If 15% of calls are repeated, the workflow pays for 115% of its tokens, and the model says so.
Review and rework
Minutes are a cost too.
Review minutes per accepted output and rework minutes per rejected output come from the review tool where it records them, and from sampled time studies where it does not. Both are priced at the same fully loaded hourly rate as the baseline, so the comparison is fair in both directions.
In the example scenario those minutes are $44,200 of a $44,821 monthly operating cost. The tokens are the other $621.
The unit
Why per-call cost is the wrong unit.
Per-call cost flatters everything. It ignores the call that was thrown away, the second attempt, and the nine minutes a person spent redoing an output that failed review. It is also the number vendors quote, which is why pilots get funded on it.
We report two numbers instead. Cost per task includes every call and every minute the task consumed. Cost per accepted output divides by the outputs you kept, and it is the honest one: a workflow with a 50% acceptance rate pays for two tasks for every one it delivers.
| Line | How it is counted | Per month | Per task |
|---|---|---|---|
| Tokens | 82.8M in, 8.28M out, retries included | $621 | $0.052 |
| Review | 9,600 accepted × 2 min | $20,800 | $1.73 |
| Rework | 2,400 rejected × 9 min | $23,400 | $1.95 |
| Operating cost | tokens + minutes | $44,821 | $3.74 |
| Per accepted output | ÷ 9,600 outputs kept | — | $4.67 |
Step 03 · Baseline
Baseline: what a person costs
A saving is a comparison. The baseline is the other half of it, and it has to survive a finance review.
How it is set
Measured, not remembered.
Three inputs. Sampled time studies of the task done by hand, with the people who actually do it. Historical throughput from your ticketing, case, or workflow system, which says what a week of the job really produced. And the fully loaded hourly cost from finance: salary, benefits, overhead, and tooling, per hour.
We reconcile the studies against the throughput. Where they disagree, we report both and build the model on the lower one. In the example scenario, 5 minutes per task at $65 an hour is $5.42 per task, or $65,000 a month at 12,000 tasks.
Why it is often lower
The pilot deck was flattering.
Pilot decks tend to assume the slowest version of the human job: every task done from scratch, at a rate rounded up, with no batching and no templates. Measured, people batch, skip, and reuse. An honest baseline is often lower than the pilot deck assumed, which makes the saving smaller and the number defensible.
We would rather hand you a smaller saving a CFO will sign than a larger one that dies in the first budget review.
When there is no baseline
A workflow that did not exist before.
Some workflows have no human version. Nobody was summarising every support call before the model did. Then there is nothing to save against, and we say so rather than invent a person who was never hired.
Two honest options remain. Compare against the alternative that would actually be funded: a contractor, a rules-based script, or doing nothing. Or report cost per accepted output on its own, without a savings claim, and let the business decide whether the output is worth that price. Invented baselines are how a pilot gets a payback period it will never see.
Step 04 · Verdict
Verdict: keep, fix, or stop
Four numbers per workflow, a sensitivity strip, and a written recommendation. One of the three answers is stop, and we use it.
Four numbers
One line per workflow.
Every workflow gets the same four numbers, so two workflows can be compared with each other and one workflow can be compared with itself next quarter.
- Cost per accepted output. What you pay for each result you keep, tokens and minutes included.
- Payback. How many months the workflow's cash flow takes to repay the build. When it never does, the verdict says so in words, not with a symbol.
- Breakeven acceptance. The quality floor. Below this acceptance rate the workflow loses money.
- Breakeven token price. The multiplier your provider's prices could rise by before the margin is gone.
- Cost per accepted output
- $4.67 $3.74 per task; a person costs $5.42
- Payback
- 2.3 months on a $40,000 build
- Breakeven acceptance
- 64% the workflow runs at 80%
- Breakeven token price
- ×24.1 $120.50 in, $602.52 out per million tokens
Example scenario from the ROI model: invoice triage at 12,000 tasks a month. Not a client result.
The sensitivity strip
Quality dominates price.
Every verdict ships with a sensitivity strip: net monthly savings at half and double the volume, with acceptance 15 points lower, with token prices doubled, and with review time doubled. The strip says which lever matters, and it is rarely the one the vendor is talking about.
In the calculator's example scenario the workflow nets $14,346 a month. Double the token price and it barely moves, to $13,725. Double the review time and it turns into −$6,454. “Fix” almost always means the quality lever, acceptance rate and review minutes, not the price lever. Run the example in the ROI model and move the inputs yourself.
| Scenario | What changes | Net / month | vs base |
|---|---|---|---|
| Base | as entered | $14,346 | — |
| Volume ×0.5 | 6,000 tasks a month | $4,256 | −$10,090 |
| Volume ×2 | 24,000 tasks a month | $34,525 | +$20,179 |
| Acceptance −15 pts | 65% instead of 80% | $696 | −$13,650 |
| Token price ×2 | $10 in, $50 out per million tokens | $13,725 | −$621 |
| Review time ×2 | 4 min per accepted output | −$6,454 | −$20,800 |
Three outcomes
Keep, fix, or stop.
- Keep. Ahead at today's inputs, with margin left on every row of the strip. It goes on the monitoring list and gets its four numbers again next quarter.
- Fix. Ahead or behind by a number you control: acceptance rate, review minutes, retries. The verdict names the number, the target, and the date to measure again.
- Stop. No acceptance rate makes it pay, or the baseline was never there. We write that down, in the same report and the same font as the wins.
Stop is a verdict, not a failure of the engagement.
Afterwards
What you are left owning
Nothing in the engagement depends on us staying.
Yours to keep
Four things, all in your stack.
- Instrumentation running in your stack. The tagging, the roll-up queries, and the dashboard, maintained by your engineers from the runbook.
- The model as formulas, not a slide. A spreadsheet-portable set of formulas, the same ones behind the public ROI model, so finance can open it, audit it, and change an input.
- A quarterly re-verdict cadence. Prices, models, and volumes move. Every workflow gets its four numbers again each quarter, on a date in the calendar, run by your team or by us.
- People on your side trained to rerun it. Whoever owns the AI budget and whoever owns the platform can produce the verdict without us. If they cannot, we have not finished.
Start with one workflow.
Send us the workflow you are least sure about and last month's token bill. We come back with a scoping note that says whether an engagement makes sense. Sometimes it says it does not.