What happens after an AI proof of concept works?

Start with one complete workflow

Choose a narrow task with a clear beginning and end. For an invoice workflow, that might mean receiving a document and creating an approved draft accounting entry. Payment stays outside the pilot.

1. 01 · RECEIVEKeep the original invoice.
2. 02 · CHECKExtract fields and validate records.
3. 03 · REVIEWRoute exceptions for approval.
4. 04 · RECORDCreate one approved draft entry.

Walk through the whole path with the person who does the work today. Include the spreadsheet they consult, the supplier they call, and the corrections they make before an entry is ready. Each of those details can change the implementation.

You may only need AI for reading an inconsistent document. Validation rules, permissions, and approvals can remain explicit. Anthropic's guide to building effective agents also recommends starting with the simplest approach that meets the task.

1. Decide where exceptions go

An unreadable attachment needs a different response from a missing purchase order. A changed supplier bank account needs a different review from a spelling mismatch.

List the cases that should stop processing. For each, record the reason, the evidence a reviewer needs, and the person responsible. Give the reviewer a way to correct, reject, or return the item without losing the original.

Test those cases deliberately. Keep a small evaluation set containing ordinary invoices and difficult examples, with the expected outcome written down. Re-run it when the model, prompt, or integration changes. A confident-looking answer is not evidence that the fields are correct.

Anthropic's guide to agent evaluations explains how repeatable tasks and explicit success criteria help teams catch regressions as a system changes.

Before moving on

Send an unreadable document through the workflow. Confirm that it reaches the right queue with an explanation and that a person can resolve it.

2. Define what can happen without approval

Reading an invoice, preparing an entry, changing supplier details, and releasing money are separate actions. Grant access for the action the pilot needs, and enforce that boundary in the connected systems.

For this example, the workflow can prepare a draft. A reviewer checks the source document and approves that specific draft before it is recorded. If the draft changes after approval, it needs review again.

Treat instructions inside invoices or emails as untrusted document content. A sentence in an attachment must not grant the workflow new permissions or change the approval policy. Log which person approved which action, while limiting access to sensitive document data.

These boundaries follow OWASP's guidance on excessive agency: minimise permissions, require approval for high-impact actions, and enforce authorisation in downstream systems.

Before moving on

Attempt an action outside the pilot's scope. Confirm that the system blocks it even if the model requests it.

3. Make retries safe

Suppose the accounting system accepts an entry but the connection drops before it confirms success. Your workflow sees a timeout. Sending the same request again could create a duplicate.

Give each intended write a stable operation identifier and use the destination's duplicate-prevention mechanism where available. Keep the same identifier when retrying that operation. AWS explains this pattern in Making retries safe with idempotent APIs.

If the destination cannot guarantee that behaviour, reconcile its records before another write. Route an uncertain result to a person. Separately, check for the same invoice arriving through different emails; duplicate documents and repeated API calls are different problems.

Before moving on

Test both a duplicate submission and a timeout after a successful write. Confirm that the workflow creates one intended entry and leaves an understandable record of what happened.

4. Name the person who owns Monday morning

Someone needs to notice a growing review queue, an expired connection, or a sudden increase in corrections. Name a business owner for the outcome and a technical owner for the system. In a small team, one person may hold both roles, but the responsibilities still need to be clear.

Agree when alerts need a response and who covers absences. The handover should include access, a short operating guide, a change log, and a tested way to pause new work.

Google's SRE chapter on monitoring explains why alerts should identify clear problems that people can act on, with enough context to investigate them.

Pausing the workflow does not undo entries it already created. Define how to identify affected records, reconcile them, and make authorised corrections. Keep the existing manual process available during the pilot.

Before moving on

Ask the operator to pause the workflow and resolve a failed item using the guide. Watch where they need help, then improve the handover.

5. Measure the work that remains

Record the current process before rollout: handling time, correction rate, queue age, and completed volume. Compare similar work over an agreed pilot period. A faster extraction step tells you little if reviewing its output takes longer.

Count human review, exception handling, software, model usage, maintenance, and the initial implementation effort. Track failure and correction rates alongside speed so that a throughput increase does not hide more rework.

Freed capacity has value, but it becomes a cash saving only when spending changes. Be explicit about whether the goal is less overtime, more completed work, fewer corrections, or a shorter turnaround. Avoid counting the same benefit twice.

Before moving on

Agree what result would justify expanding the pilot, what would require another iteration, and what would make you stop. Choose those thresholds before seeing the results.