How to Choose an AI Chatbot for Your Website: 12 Questions to Ask Before You Pay

To choose an AI chatbot for your website, ignore the feature grid and ask how each tool behaves in the situations that cost you money or trust: when it doesn't know an answer, when a visitor wants a person, when you hit your plan's limit, and when someone in the EU or on a screen reader uses it. The 12 questions below cover those, and a 15-minute test at the end shows you the real answers.

I build ChatBento, so I'm not neutral. I've tried to write questions that are fair to any tool, including the ones where ChatBento isn't the best fit.

Answers

1. What does it learn from, and can I control it?

A support chatbot is only as good as the content it reads. Ask:

• Can it read my website by itself (following links and the sitemap), or do I have to upload everything?
• Can I add Q&As and text for things my site doesn't say?
• Can I see what it learned, and remove pages that are outdated?

The last one matters more than it sounds. An old blog post with last year's prices is the most common source of wrong answers. I wrote a checklist for this in how to train an AI chatbot on your website.

2. What does it do when it doesn't know?

This is the most important question on the list. A good support bot says "I don't have that information" and offers a way to reach you. A bad one invents an answer that sounds right.

Ask the vendor directly whether the bot answers only from your content or also from general knowledge. Then test it (see the end of this post). An invented refund policy can cost you real money: an airline learned that in court, which I covered in the Air Canada chatbot ruling.

3. Does it stay on topic?

Visitors will ask your bot for poems, homework help and, occasionally, to ignore its instructions. Check that it politely declines questions unrelated to your business, and that it can't be talked into taking actions it shouldn't. A bot that only answers questions and has no access to your other systems has much less that can go wrong.

People and leads

4. How does it hand off to a human?

Every bot hits questions it can't answer. Find out what happens next:

• Does it collect the visitor's name and email?
• Where does that go: your inbox, a helpdesk, a dashboard nobody opens?
• Does the visitor know when they'll hear back?

A handoff that leads nowhere is worse than no chatbot.

5. Can I read the conversations?

You'll learn more from a month of transcripts than from any report: the questions customers really ask, the pages that confuse the bot, the gaps in your content. Make sure you can read every conversation, not just a summary.

Price

6. What exactly is one billable unit?

AI chatbot pricing comes in three shapes: per resolved conversation, per message credit, or a flat monthly fee with a cap. They count different things, so a price per month tells you little until you convert it to your volume. Per-resolution, credits or flat fee? works through the math at three volumes.

Ask: is one unit a reply, a conversation or an outcome? Do some AI models use more of my allowance than others? Are test chats billed?

7. What happens when I hit the limit?

The answer varies: extra charges, automatic top-ups, or the bot simply stops. Ask what a visitor sees on the day you run out. In ChatBento, the widget stops writing AI answers at the cap and shows the contact form instead, so visitors can still reach you and you aren't charged extra.

8. Can I try it properly before paying?

A free plan, a free trial and a demo are different things. A trial lets you test with your real content; a demo shows you someone else's. Check whether a card is required and what happens when the trial ends.

For reference: Chatbase has no free plan, only a 7-day trial, and its Hobby, Standard and Pro plans are $40, $150 and $500 a month (checked October 20 and 21, 2026). ChatBento has a free plan with 50 AI messages a month, a 14-day trial with no card, and paid plans at $8, $30 and $100. The side-by-side comparison covers where Chatbase is the better choice too.

Your website

9. How does it install on my platform?

Most website chatbots install with one script tag. The question is where it goes on your platform: Framer, Webflow, WordPress, Shopify, Wix and Squarespace each have their own custom code setting. Ask for a guide for your platform, and check whether you need a paid plan on your site builder to add custom code.

10. Will it slow my site down?

A chat widget loads on every page, so its weight matters. Ask how big the script is, whether it loads asynchronously (after your page), and whether it can clash with your site's styles. Then check your pages in PageSpeed Insights before and after installing. I go into the details in chat widgets and Core Web Vitals.

Compliance

11. Does it say it's an AI, and is it accessible?

Since August 2, 2026, the EU AI Act requires that people are told they're talking to an AI, clearly and from the first interaction. Check that the widget labels itself in plain text, and that your greeting doesn't claim to be a person. More in the EU AI Act and website chatbots.

While you're at it, try the widget with only a keyboard: can you open it, type, send and close it with Escape? The full list is in is your chat widget accessible?.

12. Where does the data go?

Your visitors type real questions, names and emails into the chat. Ask:

• Which companies process the conversations (hosting, the AI model provider, email)?
• Does the AI provider train its models on the conversations?
• How long are conversations kept, and can I delete them?

If you sell to customers in the EU or UK, you'll need these answers for your own privacy notice anyway.

The 15-minute test

Answers from a sales page are a start. A short test with your own content tells you more:

1. Write down the ten questions your visitors ask most, in the words they use.
2. Add three questions your site doesn't answer, and two that have nothing to do with your business.
3. Set up a trial or free plan on each tool you're considering and point it at your website.
4. Ask every bot all fifteen questions.
5. Score them: correct answers, honest "I don't know" replies, and anything invented.
6. Ask one bot to hand you to a human and check where the lead actually arrives.

Then compare the monthly price at the message volume you expect, not the headline price.

A scorecard you can copy

If you'd like ChatBento in the comparison, it has a free plan and install guides for every major site builder. And if you're comparing it with Chatbase in particular, start with ChatBento vs Chatbase.