How to evaluate AI vendors: a buyer's checklist for SMBs

1. Data: where does it go, who sees it, can you pull it back?

Before any feature conversation: who hosts the data, is it used to train their models, what's the deletion SLA, and do you get an export. A vendor that can't answer those in writing is a vendor that hasn't thought about it.

2. Real ROI, not "could save up to"

Ask for two case studies the size and shape of your business. Ask for the specific metric that changed and the baseline it was measured against. "We reduced ticket handle time" is not a metric. "Average first-response time went from 9 hours to 22 minutes across 12,000 tickets/month" is.

3. Demo red flags

• The demo uses their data, not yours, and they decline to run it on a sample of yours.
• They show "the agent" but won't show the prompts, tools, or fallbacks behind it.
• They cannot articulate when the system will be wrong - every model is wrong sometimes.
• Pricing is "let's get on a call" with no public floor.
• They oversell autonomy. "Set it and forget it" rarely survives contact with real customers.

4. Lock-in: what breaks if you leave?

Three sub-questions: (a) Do you keep your data, including the embeddings, knowledge bases, and conversation history? (b) Can you replicate the workflow elsewhere, or is it locked to their proprietary builder? (c) What's the notice period and renewal mechanic? Auto-renew with a 90-day window is a red flag.

5. Pricing that survives growth

AI pricing is messier than SaaS pricing was. You'll see per-seat, per-conversation, per-token, per-minute, per-resolution. Model what your bill looks like at 3x and 10x current usage before you sign. A great deal at 100 calls/month can be unworkable at 10,000.

6. The integration question

Where does the output land - directly in the system your team already works in, or in a new dashboard nobody will open? If a tool requires a new daily destination for your team, the bar for value gets much higher. The best AI vendors disappear into the tools you already use.

7. Contract terms worth pushing on

• Month-to-month or quarterly for the first year. Lock in only after value is proven.
• No price escalation above CPI without 60 days notice.
• Data export on demand and within 30 days of termination.
• SLA with actual remedies, not "best effort."

Run this checklist, and 60% of vendors will self-eliminate before the second meeting. That's the point. The ones that survive it tend to be the ones still useful 18 months in.