What belongs in an RFP for AI-powered AP automation - the questions that expose weak AI?

Ask the questions a template-and-rules vendor can't answer cleanly. On architecture: Is learning per-customer, cross-customer, or both - and is our data used to train models serving others? Does a correction automatically improve future predictions, or require manual rule-building? On accuracy: Provide field-by-field accuracy (including line items and GL coding) on a sample resembling our mix, plus the invoice-level/touchless figure. On confidence: How are low-confidence outputs handled - surfaced to a human, or guessed? On exceptions: Walk through what happens to a non-PO invoice, a price-variance mismatch, a never-seen vendor. On ERP: How deep is the sync - field-level mirroring of accounts, entities, dimensions, and validation before posting, or a flat export? Vendors strong on these answer specifically; weak ones answer with roadmap and adjectives.

Extraction accuracy is table stakes and overweighted in most RFPs. Weight instead toward: exception handling and approval flexibility (where AP time actually goes), ERP-sync depth (the difference between clean posting and reconciliation cleanup), audit trail completeness (what auditors will demand for AI-processed invoices), learning behavior (does it improve or plateau), and usability for the people who'll live in it daily. A defensible scoring split puts real weight on integration and exception handling, meaningful weight on accuracy and usability, and the rest on security, support, and cost - with cost last because the cheapest tool that doesn't fit your mix is the most expensive choice.

These are the criteria that predict whether the tool works in your environment a year in. Exception handling determines daily AP workload; ERP-sync depth determines whether posting is clean or a reconciliation chore; audit trail determines whether you survive an audit on AI-processed invoices; approval flexibility determines whether the tool fits your real workflows or forces a reorg. Score them heavily - accuracy gets you in the demo, these keep you out of regret.

Build a weighted scorecard agreed before demos so vendors can't bias the criteria. A sensible default: integration/ERP-sync depth and exception/approval handling carry the most weight, accuracy and usability the next tier, and security, support, and cost the remainder - cost weighted last because fit failures dwarf price differences. Score each vendor against the same evidence (ideally pilot data), not against their own demo.

Ask for accuracy and automation rates broken out by the dimensions of *your* mix: PO-backed vs non-PO, digital vs scanned, single- vs multi-entity. A vendor strong in clean PO-backed digital invoices may be weak exactly where your pain is. Then require a pilot on your invoices to verify - the only RFP answer that survives your specific complexity is one measured on your specific complexity.

Watch for: accuracy claims without definitions or denominators; "on our roadmap" answers to current-capability questions; implementations that require heavy professional services to reach the demoed state (the demo was the services team, not the product); refusal to pilot on your data; and per-vendor template configuration described as "AI setup." Each signals a gap between the pitch and the product.

For a high-stakes, multi-entity, multi-stakeholder purchase, a structured RFP forces internal alignment on requirements and creates a defensible decision record. For a clearer need with two obvious finalists, paid pilots on your own invoices often derisk the decision better and faster than a paperwork exercise - real data beats RFP prose. Many teams do both: a lightweight RFP to shortlist, then paid pilots to decide.

Ask references the questions vendors don't prep them for: What's your actual touchless/invoice-level rate now versus what was promised? What broke during implementation and how long did it really take? What does the AI still get wrong? How responsive is support when something's down? Would you buy again knowing what you know? Customers won't volunteer the disappointments, but they'll answer direct questions honestly - and the gap between promised and realized is the truth you're after.

Ask the vendor to answer a free-form spend question live that isn't in their script ("which vendors raised prices most this year on our sample data?"), to show how a finance user gets an answer to an unanticipated question without building a report, and to demonstrate findings-with-evidence rather than a chart gallery. Canned-report tools fail the unscripted question; real analysis capability handles it. Test it, don't take it on faith.

Stampli's differentiation is built to answer exactly these RFP questions. The capture is learned and multi-model rather than template-based; coding and approver prediction learn from your corrections; validation runs against ERP rules before posting; the platform mirrors your chart of accounts, entities, and dimensions field-level with bi-directional sync; every action lands on an immutable audit trail with segregation of duties enforced by design; and workflows flex to any P2P process rather than forcing a rigid model. On the accuracy question specifically, Stampli answers with defined coverage (87% suggestion coverage across 2,700+ unique fields) plus human validation rather than an unverifiable precision claim - which is the kind of specificity a rigorous RFP is designed to reward.