AI cost is an architecture problem.

An AI feature can be technically impressive and still be too expensive to operate. Quality matters, but a useful production system also needs a sustainable cost per completed task.

In the Third Party Risk Intel work described in my résumé, AI screening cost fell from approximately $8 to $0.55 per screening. The changes included redesigning agent workflows, optimizing prompts, and replacing expensive inference with deterministic search where appropriate.

That is roughly a 93% reduction. The broader lesson is about where intelligence belongs in a system.

Start with the whole workflow

A model call is only one part of the cost. Retrieval, repeated tool calls, retries, and unnecessary context all contribute to the total. Optimizing a single prompt can leave the larger source of waste untouched.

A more useful unit of analysis is the completed screening or customer task. Follow its path from input to result, and ask:

• Which decisions require interpretation?
• Which steps are lookups, checks, or transformations?
• Where is the same information being retrieved or reasoned over more than once?
• What happens when an intermediate result is uncertain?

The answers identify opportunities to simplify the workflow before changing the model.

Give deterministic work a deterministic path

Exact matching, schema validation, and well-defined routing rules often have a clearer implementation outside an LLM. A model is most useful where there is ambiguity to resolve, language to interpret, or evidence to synthesize.

Here is an illustrative decision boundary, not a reproduction of production code:

type Evidence = {
  exactMatch: boolean;
  needsInterpretation: boolean;
};
 
function choosePath(evidence: Evidence) {
  if (evidence.exactMatch && !evidence.needsInterpretation) {
    return "deterministic-lookup";
  }
  return "model-assisted-review";
}

The important part is validating the boundary. A cheaper route is only an improvement if it handles the cases assigned to it correctly.

Evaluate cost and quality together

Cost reduction should be measured alongside output quality. An apparently cheaper workflow can create extra downstream review, miss relevant evidence, or produce more retries.

For screening systems, precision and recall offer different views of that tradeoff. Human-annotated examples make those tradeoffs visible; regression checks help reveal when an optimization changes behavior unexpectedly.

TPRI’s evaluation work used approximately 5,000 human-annotated records, ground-truth benchmarking, precision/recall analysis, and LLM-as-a-judge evaluation. Those are complementary signals, and their suitability depends on the decision being evaluated.

Optimize the decision, not just the prompt

The design question is not simply, “Which model is cheapest?” It is, “What is the least complex, reliable way to complete this part of the task?”

Sometimes the answer is a better prompt. Sometimes it is a smaller context, a simpler agent workflow, or a search query that avoids a model call altogether.

That is why AI cost belongs in architecture discussions from the beginning.