How to evaluate an AI project before production

Publisher, date and evidence

Published by Capitao.Consulting on 21 September 2026. This is our proposed evaluation method for buyers of AI implementation services. It is not a customer case study, an independent vendor comparison or a report of measured customer results.

The NIST AI Risk Management Framework is a voluntary framework for considering trustworthiness throughout AI design, development, use and evaluation. NIST also publishes a generative AI profile addressing risks specific to generative systems. Those are external references for risk assessment; the workflow and numerical example below are our own practical guidance, not NIST requirements or an endorsement.

Primary source: NIST AI Risk Management Framework and generative AI profile

Start with an acceptance decision

Write down one real task, who owns it, the permitted inputs, the expected output and what happens if it is wrong. “Prepare a draft reply using the current customer contract for a staff member to approve” is testable. “Make customer support intelligent” is not specific enough to accept or reject.

Before building, record how the current process performs. Include human preparation time, review time, corrections, missed cases and time to completion. Agree the minimum acceptable quality and the maximum tolerable cost with the process owner. There is no universal pass percentage that makes every AI task safe.

Choose automation, RAG or an agent

• Use ordinary automation when inputs and rules are stable: routing an approved invoice by department should not require a language model to invent a decision.
• Use retrieval augmented generation (RAG) when the task is to answer from a changing document collection. Check whether the retrieved passages support the answer and whether permissions apply before retrieval.
• Use an AI agent when the next action depends on interpreting context and selecting among permitted tools. Limit its tools, define an approval boundary and test recovery when a tool fails.
• Combine approaches when appropriate: deterministic validation can reject an invalid invoice total after a model extracts the fields. The model should not replace a reliable calculation.

Build a representative evaluation set

Collect examples from the actual workflow, with permission and appropriate data handling. Include routine work, incomplete inputs, ambiguous instructions, conflicting documents, outdated documents and requests outside the agreed scope. Keep examples used for final acceptance separate from examples used to tune the system.

For each example, record the expected behavior, allowed evidence and a reason for acceptance or rejection. Some tasks permit several correct answers; score the facts and the action taken instead of requiring identical wording. Have a domain expert resolve disputed judgments.

Record the model version, prompt, retrieval configuration, tool permissions and evaluation date. Repeated runs expose variability. Passing one polished demonstration does not establish reliability across the real workload.

Measure quality and cost per accepted result

Illustrative calculation only: if 100 attempts cost €20 in model and tool usage plus €80 in review, and 80 outputs are accepted, the cost per accepted result is (€20 + €80) / 80 = €1.25. This excludes implementation and fixed infrastructure costs, which belong in the full business case. It is not a Capitao.Consulting customer result or a price offer.

A faster draft is not necessarily a cheaper completed task. Compare total preparation, review and correction time with the original process at comparable quality. Report the sample size, observation period and exclusions alongside every claimed improvement.

• Accepted-result rate: outputs accepted under the agreed criteria divided by all eligible attempts. Keep failed and timed-out attempts in the denominator.
• Unsupported-claim rate: outputs containing a factual claim not supported by the permitted evidence divided by reviewed outputs.
• Correction burden: human minutes spent checking and correcting each attempt, including rejected outputs.
• Cost per accepted result: model, tool and human-review costs divided by the number of accepted results.
• Completion and latency: whether work reached the correct destination and how long users waited; record slow cases as well as the average.

Test permissions, failure and human control

Try a user who cannot access a document, an expired connection, a timeout and a repeated tool response. The system should preserve access boundaries, explain incomplete work and avoid duplicate consequential actions. Treat instructions embedded in retrieved documents as untrusted content.

For emails, payments, record changes or other consequential actions, define who may approve what and show the proposed action before execution. Confirm that refusing approval prevents the action. A human review step is useful only when the reviewer has enough context and time to detect errors.

Document which data goes to each provider, retention settings and the deployment region. Private hosting alone does not establish legal compliance; the organization must assess the complete processing arrangement and applicable obligations.

Decide whether to deploy, revise or stop

Deploy a limited rollout only after the agreed quality, cost and control criteria pass. Assign an operational owner, a fallback process and a rollback condition. Monitor the same measures after launch because real input distributions and provider behavior can change.

Revise when failures have a specific, testable cause, such as missing source coverage or an unclear tool permission. Stop or choose simpler automation when review overhead erases the benefit, the necessary data is unavailable, or the required reliability cannot be demonstrated.

Questions to ask an implementation partner

Capitao.Consulting scopes custom work around the process, integrations and operational requirements. A price or delivery date requires that scope; this guide makes no universal savings, budget or completion-time promise.

• What exactly is delivered, what is excluded, and which assumptions affect the estimate?
• Who owns the code, evaluation set and operating documentation?
• What happens when the model, integration or data source changes?
• Can we inspect the evaluation results, including failures and review time?
• Which costs continue after launch, and who handles incidents and maintenance?

Explore AI implementation services

Read the delivery approach

Company facts and buyer questions

Discuss an evaluation scope