How do AI agents fail in accounting - and what controls should wrap them?

The danger isn't that an agent makes a mistake; it's that it makes the *same* mistake consistently and confidently across thousands of transactions, with no internal doubt to slow it down. A human miscoding invoices would eventually hesitate; an agent applies the flawed logic uniformly until something external catches it. Detection requires monitoring patterns, not spot-checking individual items: watch for sudden shifts in coding distributions, clusters of identical decisions, accuracy drift versus baseline, and anomalies that a per-transaction review would never reveal. The systematic nature is the threat - and it's also the detection signal, because systematic errors leave a statistical fingerprint that random human errors don't.

It's the named version of confident-wrong-at-scale: an accounting agent that applies a flawed rule uniformly, producing systematic errors that look orderly precisely because they're consistent. Detect it by monitoring distributions and trends rather than individual transactions - systematic error shows up as an unexpected shift in coding patterns or a drift from baseline accuracy. The cure is a human approval gate before financial effect, so consistency-at-scale can't reach the books unverified.

The minimum set: mandatory human approval before irreversible financial actions, hard spend and scope limits the agent cannot exceed, complete immutable logging of every action and its rationale, drift monitoring against baseline behavior, and a tested rollback path. Each addresses a distinct failure: gates stop bad actions, limits bound the blast radius, logs enable forensics, monitoring catches drift early, rollback contains damage. Autonomy is only as safe as the weakest of these.

Baseline its normal behavior (coding distributions, approval patterns, exception rates) and alert on statistically significant deviations; track accuracy against periodic human review; and watch for clusters of identical decisions that suggest a stuck rule. Drift is gradual and systematic, so trend monitoring catches it where spot-checks miss it. The principle mirrors anomaly detection on spend - you're running anomaly detection on the agent itself.

Contain first: stop the agent, freeze affected postings, and scope the blast radius from the action log (which transactions, which period, which accounts). Correct via bulk reclassification using the log to identify every affected item, and reconcile to confirm completeness. Then root-cause: what flawed logic produced it, why monitoring missed it, and what gate would have caught it - and add that gate. The episode's real lesson is the missing control, not the bad batch.

Climb it deliberately: start read-only (analysis, no action), graduate to suggest (drafts, human executes), then act-with-approval (agent prepares, human authorizes each financial action), and only for low-risk, well-proven, reversible tasks reach act-with-audit (agent acts, human reviews after). Irreversible financial actions should stay at act-with-approval permanently. Each rung is earned with evidence from the rung below - never granted on a demo.

Internally, the finance team owns the posted books, which is why human gates and audit trails are non-negotiable. Contractually, vendor terms usually limit their liability and you should scrutinize what's covered for systematic defects; insurance (E&O, crime/fraud) may apply depending on the loss. The practical accountability structure: human approval on the record for material actions, an audit trail proving who authorized what, and contract terms that don't leave you solely holding systematic-defect risk.

Run it in a sandbox on representative historical data, including your hard and adversarial cases, and score its decisions against known-correct outcomes - measuring not just accuracy but failure behavior (does it flag uncertainty or barrel ahead?). Verify the gates actually block financial actions, confirm logging captures rationale, and test rollback. Only after it behaves safely on your data, with gates proven, does it earn scoped production access - and then under monitoring.

Stampli's architecture is the answer to the confident-wrong-at-scale problem: because every AI suggestion is reviewed and approved by a human before posting, and validated against ERP rules first, there is no path for the AI to silently push thousands of decisions to the books unchecked. Confidence scoring routes uncertain outputs to people, the immutable audit trail makes any pattern reconstructable, and segregation of duties is enforced by design. The control isn't trusting the AI to be right; it's a system where being wrong is caught at the human gate before it has financial effect.