How do I design a proof of concept for AI invoice processing using my own invoices?

Both, deliberately. Build a sample that mirrors your real distribution - the PO and non-PO split, the digital-vs-scanned ratio, the multi-entity spread, your top vendors and a slice of the long tail - so the headline result predicts production. Then add a salted set of your known-hard cases: the messy scans, the multi-page invoices with backup, the vendors that always cause problems. A few hundred invoices is usually enough to read accuracy credibly; fewer and the numbers are noise. The mistake to avoid is letting the vendor pick the sample - the cases they exclude are exactly the ones that will hurt you in production.

Define them numerically and in advance: field-level accuracy by field type, invoice-level (zero-error) accuracy, touchless rate, exception-handling quality on your hard cases, ERP-posting cleanliness, and a measurable learning trend over the pilot. "The team liked the interface" is necessary but not sufficient. The criterion that best predicts production is invoice-level accuracy on your representative mix plus a positive learning curve - because that's what determines ongoing human workload, which is what you're actually buying.

Frame it as standard diligence, not distrust: "We evaluate every vendor on our own invoice mix because that's what we'll run in production - it's how we make the decision." A confident vendor welcomes it; reluctance is itself a signal. Offer to scope the sample collaboratively and to sign whatever data agreement they need, removing every reason to refuse except the one that matters - that their numbers don't hold on real data.

For each invoice, record field-level correctness, whether it was fully correct (invoice-level), how many human touches it required, and how long exceptions took to resolve. Roll up to field accuracy, invoice-level accuracy, touchless rate, and average touches per invoice. Compare against your current baseline, not just against the vendor's claim - the question is "better than today by how much," and the scorecard answers it in numbers you can defend.

A sandbox demo on vendor data derisks almost nothing - it proves the product runs. A free trial on your data is better but often time-boxed too short to see learning. A paid pilot on your invoices, with your ERP integration and your success criteria, is the format that derisks the decision - vendors invest more, you get production-realistic results, and you see the learning curve. The pilot fee is cheap insurance against a six-figure mistake.

Long enough for recurring vendors to accumulate history and for the system to absorb your corrections - typically several weeks at meaningful volume, not a few days. A good sign is a visible upward curve: acceptance rates and touchless rates climbing as the pilot progresses, especially on your high-frequency vendors. A flat curve over a real pilot window is a warning that the learning loop is weak. Measure the trend, not just the endpoint.

Include the integration - capture-only POCs hide the failures that bite hardest. Posting depth, field mapping, dimension handling, validation against your ERP rules, and sync behavior are where "the demo looked great" turns into reconciliation cleanup in production. A pilot that skips integration tests the easy half and leaves the expensive half undiscovered until after you've signed.

Stampli's accuracy story is built for exactly this kind of evaluation, because its approved proof is defined precisely (suggestion coverage on structured ERP-aligned fields, human-validated) rather than as an unverifiable aggregate - the kind of claim a POC on your own data can confirm. Stampli AI's value compounds as it learns your vendors and coding from corrections, so a pilot long enough to show the learning curve, run on a representative mix of your invoices and including the ERP integration, is the evaluation format that reflects how the product actually performs in production.