Research into practice

Choose the AI workflow your customers can use

Start with a useful revenue report, compare complete systems and count the cost of reaching a result the customer can actually use.

Mariya Valui · Technology Entrepreneur · 6 min read

Published: 19 October 2026Updated: 19 October 2026

Suppose you want to offer customers a useful monthly revenue report. The agent needs to combine billing data with customer records, explain changes and produce a file the customer can use. Choosing the model with the highest benchmark score answers only one part of that product decision.

The model will sit inside a system: document access, tools, calculations, memory, output formats and a way to continue when something is missing. The business buys the result of that combination. I would choose the system by asking what it can deliver repeatedly on the actual task.

Why a strong model is not the whole product

AlphaEval’s preprint, version 1 dated 14 April 2026 evaluates complete agent products on 94 tasks from seven companies. Table 3 reports different scores for the same model in different products. It is a historical configuration snapshot, not a current ranking; its economic estimates are not realised customer savings.

For my own product decisions, this suggests a practical separation: model capability, workflow capability and the usefulness of the final artifact. A good answer in a chat window does not establish that the system can read the right files, preserve spreadsheet formulas or hand the report to the next process.

It also suggests an opportunity. A business may create substantial value by connecting a capable model to a well-chosen workflow. The useful innovation could be the path from fragmented records to an understandable customer decision, rather than a new underlying language model.

Define a report worth paying for

Consider an illustrative service that produces a subscription-revenue report. One export contains invoices and credits; another contains customer accounts and plan changes. The requested period is the calendar month. The customer wants to know what was billed, what was collected and which changes explain the movement.

Those questions need different measures. An issued invoice is not automatically collected cash. A refund can relate to a prior invoice. A customer’s plan can change without applying to the whole month. A useful report states its definitions before presenting a confident total.

I would define a small deliverable: a reconciled workbook, a short explanation of the main movements and a list of unresolved records. Each explanation should lead back to supporting rows. A discrepancy should be visible enough for the next process to investigate; it should not disappear into fluent prose.

This is more specific than “analyse the data”, while leaving the agent room to choose how to do the work. It defines the service’s value without dictating an untested implementation.

Compare complete approaches on the same work

I would compare a few meaningfully different systems. One might give a general agent the full task. Another might combine a deterministic data transformation with a model that explains the results. A third might use parallel agents for separate source investigations, joining their findings against one agreed period and customer definition.

The comparison needs the same source snapshot and expected deliverable. Otherwise, a system using cleaner inputs can appear stronger than a better system facing harder data. I would record the model, accessible tools and workflow version with each run so that a later change has something specific to be compared against.

The ordinary case matters, but so does a plausible exception: a credit without its original invoice, a duplicated record or a customer identifier that changed. These are proposed evaluation examples, not experiments I have conducted. They reveal whether the product can continue usefully when the work stops looking like a demonstration.

A smaller model with a reliable calculation tool may be sufficient for a stable extraction step. A more capable agent might earn its cost on ambiguous explanations or unfamiliar document structures. I would let task evidence determine that allocation, rather than force every step through one expensive model.

Spend on judgment where it changes the product

Separate execution and evaluation can help with this choice. Anthropic’s March 2026 engineering account describes a planner, generator and evaluator working on applications, with explicit criteria and feedback. It also describes added cost and latency. That is evidence for a particular engineering approach, not a requirement to attach a large agent team to every small task.

For the revenue report, exact sums and duplicate checks belong naturally in deterministic calculations. Another agent can investigate unexplained movements or critique whether the narrative answers the customer’s question. Parallel checks are useful when they contribute different evidence. Repeating the same vague quality judgment adds less.

The product must also be usable. Correct data in an unreadable workbook still creates work for the customer. A beautiful PDF without underlying calculations may be unsuitable for a customer who needs to continue the analysis. I would assess correctness, completeness and usability separately before combining them into a decision.

Measure the cost of a usable result

Cheap generation can be expensive delivery. In a hypothetical cost comparison, one run costs £0.40, requires four repair attempts at £0.20 each and a £0.10 final check: £1.30. Another costs £0.90 plus the same £0.10 check: £1.00. This arithmetic illustrates a decision, not measured performance of named products.

That does not mean the more expensive model is always cheaper overall. A reusable transformation may remove most repairs from the first workflow. A five-minute internal estimate may not justify either process. The relevant cost includes what it takes to reach a usable result and keep it usable when the inputs change.

I would begin with one recurring report, a normal example and a small set of realistic exceptions. That is enough to discover whether the biggest constraint is the model, access to the data, the workflow or the definition of the product itself. The outcome should be a clearer business choice, not a larger collection of AI scores.