How to Stop Your Website Chatbot Giving Wrong Answers
To stop your website chatbot giving customers wrong answers, train it on your own up-to-date information, give it clear instructions about what it must never guess, and test it with real customer questions. Keep reviewing its conversations, correct mistakes when you find them, and make sure customers can reach a person when they need one.
You can still have a chatbot answering routine enquiries around the clock and taking pressure off your team. The important thing is making sure it knows its limits.
Why do AI chatbots sometimes give customers wrong information?
Imagine a customer visits your website and asks whether they can return something they bought six weeks ago.
Your chatbot confidently tells them they have 60 days to return it. There's just one problem: your returns policy changed last month, and the limit is now 30 days.
The customer believes the answer. Your support team later has to explain that it was wrong.
That's exactly the sort of situation business owners are discussing on Reddit. Some have reported chatbots inventing discount codes, others have seen bots giving incorrect returns advice, and some are questioning whether AI support is worth the risk.
These concerns are reasonable.
AI chatbots can produce convincing answers even when the underlying information is incomplete, outdated or contradictory. This is sometimes called an AI hallucination, which simply means the AI has generated something that isn't supported by the facts.
There are several common reasons:
- Outdated information: The bot has learned from an old policy, promotion or pricing page.
- Missing information: A customer asks something that isn't covered in the bot's training material.
- Conflicting information: Two documents contain different answers to the same question.
- Too much freedom: The bot uses general knowledge or makes assumptions instead of sticking to your business's actual rules.
- Poor testing: Nobody checked how it would handle awkward or unusual questions before putting it on the website.
The solution isn't to expect AI to become perfect. It's to give it reliable information, sensible boundaries and regular supervision.
Start with your own information, not the AI's general knowledge
The first step towards more reliable answers is deciding where your chatbot gets its information.
For customer support, your business's own website, policies and documents should normally be the starting point.
A customer asking about your membership fees doesn't need a general explanation of how membership organisations work. They need to know what your organisation charges.
Similarly, someone asking about a hotel's cancellation policy needs that hotel's actual terms, not what hotels generally do.
With FastBots, you can train a chatbot using your website. Its crawler scans website pages and sitemaps to collect information. You can also upload supporting documents, including PDFs, DOC, DOCX and CSV files.
This means you can give the chatbot access to information such as:
- Current prices, services and opening hours.
- Returns, refunds and cancellation policies.
- Membership fees, renewal dates and benefits.
- School admissions information and term dates.
- Booking procedures and frequently asked questions.
FastBots also lets you control where the chatbot looks for answers through Answer Source Settings.
You can choose Uploaded Information (your own content), ChatGPT's general knowledge, Internet Search, or a combination. There's also an option such as "Use My Content First".
For a support chatbot where accuracy about your business matters most, use your own uploaded information rather than relying on general AI knowledge or internet searches.
You can find these options in the FastBots Answer Source Settings guide.
If you're starting from scratch, our guide on how to train a chatbot on your own data explains the process.
Remove old information before it becomes a problem
Uploading the latest policy isn't enough if old versions are still sitting in your chatbot's knowledge.
For example, if you've changed your refund period from 60 days to 30 days, check whether the old policy appears in previous PDFs, archived web pages, FAQs or manually added answers.
Remove or replace outdated material wherever it's stored.
Choose one approved source for important information, keep it current, and make sure your chatbot isn't being asked to choose between conflicting versions.
Tell your chatbot what it must never guess
Having the right information is important. But your chatbot also needs instructions about how to behave when it doesn't know something.
This is particularly important for prices, discounts, refunds and policies.
In FastBots, the AI Instructions section under Tune AI lets you explain the chatbot's role, what it's supposed to help with, and what it must avoid doing.
For example, you might give it instructions like these:
Answer customer questions using the information provided by our business. Do not invent discount codes, prices, refund conditions or exceptions to our policies. If the answer is not available or is unclear, explain that you cannot confirm it and direct the customer to our support team. Never promise a refund or special arrangement that isn't stated in our information.
These are illustrative instructions you can adapt to your organisation.
You should also set a sensible default response for questions the chatbot cannot answer.
Something as simple as this is often better than a confident guess:
I don't have enough information to confirm that. Please contact our team so we can check it for you.
The wording can be friendly without pretending to know something.
See the FastBots AI Instructions guide for help setting these rules.
Keep AI Creativity low
FastBots also has an AI Creativity setting, which influences how freely the chatbot generates its answers.
For a customer support chatbot, FastBots recommends keeping this setting between 1% and 4%.
This can help encourage more controlled responses rather than imaginative ones.
However, a low creativity setting doesn't guarantee factual accuracy. It still needs good information and clear instructions.
You can learn more in the AI Creativity settings guide.
Correct wrong answers and fill in missing information
Even with careful setup, you'll probably discover questions your chatbot doesn't handle well.
Perhaps someone asks about an unusual membership arrangement, a specific product return or a course requirement that isn't clearly explained on your website.
Rather than hoping the chatbot will work it out next time, give it an approved answer.
FastBots includes a Q&A page where you can add specific questions and their correct answers.
For example:
Question: Can I use a discount code after it has expired?
Answer: Our promotional codes can only be used within their stated validity period. Please contact our team if you have a question about a particular promotion.
This is an illustrative example, not a FastBots policy.
The Q&A page also shows questions the chatbot couldn't answer, with a "View Chat" option so you can see what the customer actually asked.
You can then add the correct answer for future conversations.
On the Business plan and above, Knowledge Assistant also flags unknown questions so your team can address gaps in the chatbot's knowledge.
This creates a practical improvement process: review what customers ask, identify missing information, provide an approved answer and test the result.
See the Q&A guide and unanswered questions guide.
Remember to review these manually added answers when your policies change. An answer that was correct last month might not be correct today.
Test your chatbot with difficult questions before launch
Don't just ask your chatbot easy questions that you already know it can answer.
Try to catch it out.
For example, if you run a hotel, ask whether a guest can cancel a non-refundable booking. Then ask the same question in a different way.
If you run a school, try questions about admissions deadlines, special circumstances and fees.
If you sell software, ask about a feature that isn't available or a promotional price that has expired.
You're checking whether the chatbot answers correctly, admits uncertainty or makes something up.
FastBots has a Test Your Bot page where you can compare up to four chat windows side by side, using different AI models, creativity settings and prompts.
This can help you see which setup follows your instructions more reliably.
Try at least a few questions in each of these categories:
- A straightforward question with a clear answer.
- A question where the relevant information has recently changed.
- A question that isn't covered in your knowledge.
- A question asking for an exception to a policy.
- A question that combines two or three separate issues.
Don't just check whether the response sounds professional. Compare its claims with your approved information.
Use the Test Your Bot guide to get started.
Review real conversations after your chatbot goes live

Testing doesn't stop when you put your chatbot on the website.
Real customers have a habit of asking questions in ways nobody anticipated.
They might leave out important details, combine unrelated questions or use wording that isn't in your FAQs.
FastBots includes Chat History, where you can review and export conversations.
Make a habit of checking a sample of recent chats. Look for incorrect answers, unsupported promises, repeated fallback responses and questions that customers have to ask several times.
Pay particular attention to conversations about money, eligibility, deadlines and policies.
The Chat History guide explains how to review previous conversations.
Keep your website changes and chatbot knowledge in sync
Outdated answers are a particular problem when information changes frequently.
For example, a training provider might update its course fees. A hotel might change its check-in times. A membership organisation might introduce new renewal rules.
FastBots Auto Retrain, available on the Business plan and above, automatically revisits selected website pages to look for changes.
This can help keep information current without requiring you to manually retrain the bot each time.
However, it's still worth checking important updates yourself. Auto Retrain doesn't remove the need to review old uploaded documents, Q&A entries or conflicting information elsewhere.
Whenever a significant policy changes, update the relevant sources, refresh the chatbot's knowledge as needed and test the affected questions again.
Which customer questions should always go to a human?

Not every question belongs in an automated conversation.
A chatbot is well suited to answering everyday questions about opening hours, services, membership options, event dates and published procedures.
But some situations need human judgement, authorisation or care.
Consider directing these enquiries to a person:
- Refunds and disputes: Especially when someone requests an exception, challenges a decision or believes they've been given incorrect information.
- Safeguarding or welfare concerns: Particularly for schools, colleges, churches and organisations supporting vulnerable people.
- Legal, financial or medical advice: Where someone needs advice about their individual circumstances rather than general published information.
- Complaints and sensitive situations: When a customer is distressed, angry or describing a serious problem.
- Special arrangements: Requests involving discounts, contracts, payment terms or commitments that require approval.
- Complex account problems: Where a customer's specific details need to be checked before providing an answer.
For example, a school chatbot can explain the published admissions process. But an enquiry involving a child's individual circumstances may be better handled by school staff.
Similarly, a charity chatbot can explain how to access its services without attempting to assess someone's sensitive personal situation.
FastBots offers Live Chat on the Business plan and above, allowing a human team member to take over a conversation during configured availability hours.
There's also an Unanswered Questions Threshold. You can set how many fallback replies occur before the chatbot offers the option of speaking to a live agent.
See the Live Chat guide.
When nobody is available, the chatbot should avoid implying that a person is standing by. Give customers a clear way to contact the team instead.
For more on deciding which conversations should involve a person, read Human in the Loop and our comparison of chatbots versus live chat.
What does a reliable chatbot look like in practice?
The right setup depends on the organisation and the questions people ask.
Here are some illustrative examples.
A membership organisation: The chatbot answers published questions about joining, subscription fees, events and member benefits. It sends unusual eligibility cases or disputed renewal decisions to the membership team. See our guide to chatbots for membership organisations.
A school or college: It provides term dates, admissions information and published procedures while leaving sensitive pupil-specific matters to staff. Find out more about AI chatbots for schools.
A nonprofit or church: The chatbot explains opening hours, activities, volunteer opportunities and how to contact the right person, without making promises about individual assistance. Explore AI chatbots for nonprofits.
A software company: It answers questions from current product documentation but avoids inventing features, subscription offers or technical fixes.
A web design agency: It trains each client's chatbot on that client's approved information, tests policy-related questions and agrees who is responsible for reviewing changes. See FastBots for web designers.
In each case, the aim is the same: answer routine questions consistently while keeping the people responsible for the organisation in control.
Practical checklist: How to stop your chatbot giving wrong answers
If you're setting up a website chatbot, or reviewing one that's already live, work through these steps.
- Check your existing information. Review your website, FAQs, prices, policies and documents. Remove outdated versions and resolve conflicting information.
- Train the chatbot on approved content. Use your own website and relevant documents rather than expecting general AI knowledge to provide business-specific answers.
- Choose the right answer sources. In FastBots, select your own uploaded information when you want answers grounded in your business content.
- Write clear AI Instructions. Tell the chatbot not to invent prices, discounts, policies, commitments or exceptions. Set an appropriate fallback message.
- Set AI Creativity conservatively. Consider FastBots' recommended 1%–4% range for customer support, while remembering this isn't a guarantee of accuracy.
- Test real and difficult questions. Include expired promotions, changed policies, missing information and requests for exceptions.
- Decide when a human should step in. Establish which subjects need personal attention and how customers should contact your team.
- Review conversations and correct mistakes. Use Chat History and the Q&A page to find and address problems. Check important updates after changing your content.
If you're launching your first chatbot, our first-week website chatbot guide covers the broader process of getting it ready for visitors.
The checklist is also worth revisiting whenever you make significant changes to your business.
Frequently asked questions
How do I stop my AI chatbot giving answers from an old policy?
Update the policy at its original source, remove outdated copies from the chatbot's knowledge and refresh the affected training information. Check any saved Q&A answers that refer to the old policy.
If you use FastBots on the Business plan or above, Auto Retrain can revisit selected website pages for changes. You should still test the chatbot with questions about the updated policy afterwards.
Can I make my chatbot answer only from my own business content?
Yes. In FastBots, Answer Source Settings lets you choose Uploaded Information as the source for answers, rather than using ChatGPT's general knowledge or Internet Search.
This is a sensible choice when customers need information specific to your business. It reduces dependence on outside knowledge, although you still need to check the quality and currency of your content.
Can a chatbot still make up discount codes or incorrect prices?
Yes. AI-generated replies can still contain mistakes, particularly when the underlying information is incomplete or unclear.
To reduce that risk, use current pricing and promotion information, explicitly instruct the chatbot not to invent discounts or prices, and test questions about expired offers and unavailable codes.
For important purchases or exceptions, direct customers to your team for confirmation.
Is it worth putting an AI chatbot on my website if it can make mistakes?
It can be, especially if your team spends a lot of time answering the same straightforward questions.
A properly maintained chatbot can help visitors find published information outside office hours, while allowing your team to focus on more complex enquiries.
The important question isn't whether the chatbot can answer everything. It's whether it answers the questions you've assigned to it reliably enough, recognises when it needs help, and has a sensible way of involving a person.
Start with a chatbot you can control
You don't have to give an AI chatbot complete freedom to speak on behalf of your business.
With FastBots, you can train it on your own website and documents, control where its answers come from, give it clear instructions, test different settings and review conversations.
On the Business plan and above, you can also use Auto Retrain, Knowledge Assistant and Live Chat to help keep information current and bring staff into conversations when needed.
FastBots offers a 14-day free trial, including one chatbot, with no credit card required. Paid plans start at $39/month for Essential, with Business at $89/month.
The best place to begin is with the questions your customers ask most often. Give your chatbot the right answers, test how it handles the difficult ones, and build from there.
Start your FastBots free trial and see how it handles your own website's questions.