AI Chatbots Explained: How They Work and When to Use One
A practical guide to AI chatbots: how they work, scripted bots vs RAG assistants vs AI agents, what they cost to run, and how to tell whether your business needs one.
AI Chatbots Explained: How They Work and When to Use One
A practical guide to AI chatbots: how they work, scripted bots vs RAG assistants vs AI agents, what they cost to run, and how to tell whether your business needs one.
An AI chatbot is software that holds a conversation in natural language. Modern ones are built on large language models (LLMs): the model reads the conversation and writes the next reply. What makes one useful for a business is what surrounds the model.
The three kinds you will meet
Scripted chatbots follow a decision tree someone wrote. They cannot handle a question the tree did not expect, but they never improvise.
LLM chatbots with retrieval (RAG) search your documents first and give the model the relevant passages, so the answer comes from your content rather than the model's general knowledge. This is what most businesses mean today by an "AI chatbot".
AI agents can also use tools — look up an order, create a ticket, change a booking. They are more capable and need more care. See AI agents.
How a RAG chatbot answers a question
- The question is turned into a search over your content (vector and keyword search together work best).
- The most relevant passages are selected.
- The model writes an answer using only those passages, ideally with the source.
- If nothing relevant was found, a well-built assistant says so or hands over to a person.
What it costs to run
Running cost depends on conversation volume, the model chosen and how much text is sent per answer. Smaller models, short focused context and caching of common answers keep it low. Measure cost per conversation from the first week.
Signs you need one — and signs you don't
You probably benefit if the same questions arrive every day, answers exist in documents somewhere, and customers wait for replies. You probably don't if volumes are low, every case is unique, or answers require judgement a person must own.
Building one for your business
If you want a chatbot connected to your content and systems, see AI chatbots for business.
FAQ
Is ChatGPT a chatbot?
ChatGPT is a general-purpose assistant built on a large language model. A business chatbot usually uses a similar model but is restricted to your content, your tone and the actions you allow.
What is RAG?
Retrieval-augmented generation: the system retrieves relevant passages from your content and gives them to the model, so answers are grounded in your own information.
Can a chatbot replace customer-service staff?
It can take the repetitive questions and free staff for the cases that need a person. Designing a clean hand-off is part of building it well.