Most support teams discover the same thing when they read through a month of tickets: a handful of questions – where's my order, how do I reset my password, can I change my plan, what's your return policy – make up a large share of the volume. Those questions have clear answers, written down somewhere. An AI chatbot for customer support is, at its best, a fast way to get those answers to customers at 3 a.m., and to hand everything else to a human with the context already gathered.
At its worst, it's a confident machine that invents refund policies. The difference comes down to how you build it. Here is the step-by-step process we follow.
Step 1: Decide what the chatbot should and shouldn't handle
Start with data, not technology. Export a few months of tickets and group them by topic. For each group, ask:
- Is the answer the same for everyone, or does it depend on the customer's account?
- Is the answer documented?
- What's the cost if the bot gets it wrong?
This gives you three buckets. Answer – general questions with documented answers (shipping times, policies, how-tos). Answer with lookup – questions that need account data (order status, subscription details), which require integrations. Hand off – complaints, refunds above a threshold, legal issues, anything emotional or high-stakes.
Launch with the first bucket. Add the second once the foundations work.
Step 2: Build a clean knowledge base
The chatbot can only be as accurate as the content you give it. Before writing a line of code, audit your help centre:
- Remove outdated and contradictory articles.
- Write down answers that currently exist only in agents' heads.
- Keep articles focused – one question per article works better than long pages covering everything.
- Note who owns each article and when it was last reviewed.
This step is unglamorous and usually the most valuable. Teams often find their help centre had been quietly wrong for months.
Step 3: Choose the architecture – retrieval-augmented generation
Don't rely on a language model's general knowledge for your policies. The standard approach is retrieval-augmented generation (RAG):
- Split your knowledge base into passages and convert each into an embedding – a numerical representation of its meaning – stored in a vector database or search index.
- When a customer asks a question, retrieve the most relevant passages.
- Send the question plus those passages to the language model with instructions to answer only from the provided content.
- Return the answer, ideally with a link to the source article.
Hybrid search – combining keyword matching with semantic similarity – tends to work better than either alone, especially for product names and error codes.
Step 4: Write the system prompt and guardrails
The system prompt defines the bot's behaviour. It should cover:
- Scope: which topics it handles, and that it should hand off anything else.
- Grounding: answer only from the provided content; if the answer isn't there, say so and offer a human.
- Tone: match your brand – friendly, concise, no promises it can't keep.
- Hard limits: never promise refunds, discounts or legal outcomes; never ask for passwords or full card numbers.
Back the prompt with checks in code, because prompts alone aren't enough. Filter outputs for things like discount codes or prices that aren't in your catalogue, restrict which tools the bot can call, and treat anything a customer types as untrusted input that may try to override instructions.
Step 5: Connect to your systems (carefully)
For "answer with lookup" questions, the chatbot needs tools: look up an order, check a subscription, create a ticket. Design these with care:
- Authenticate the customer before exposing any account data.
- Read-only first. Order status is low risk; cancelling an order is not.
- Validate inputs in code – the order must belong to the logged-in customer.
- Require confirmation for any change, and log every action.
Step 6: Design the human handoff
A good handoff is what makes customers forgive the bot's limits. It should:
- trigger on explicit request ("talk to a person"), repeated failure, negative sentiment or sensitive topics;
- pass the full conversation and any looked-up data to the agent, so the customer doesn't repeat themselves;
- set honest expectations about response times outside business hours.
Never trap customers in a loop. An always-visible route to a human costs little and protects trust.
Step 7: Test before you launch
Build a test set of real past questions – a few hundred if you have them – including tricky ones: ambiguous wording, typos, questions in other languages, off-topic requests and attempts to make the bot say something it shouldn't. Run every version of the bot against it and review the answers. Then run a soft launch: put the bot in front of a small share of traffic, or let agents use it internally to draft replies first.
Step 8: Measure and improve after launch
| Metric | What it tells you |
|---|---|
| Resolution rate | Share of conversations solved without a human |
| Handoff rate and reasons | Where the bot falls short |
| Customer satisfaction on bot chats | Whether resolution is real or customers gave up |
| Answer accuracy (sampled review) | Whether it's telling the truth |
| "No answer found" queries | Gaps in your knowledge base |
| Cost per conversation | Model and infrastructure spend |
Review a sample of conversations every week. The "no answer found" list is your content roadmap. Set up alerts for spikes in handoffs or errors – they often signal an outage or a policy change the bot doesn't know about.
Build vs. buy
Many helpdesk platforms now include AI chatbot features that work well for straightforward knowledge-base answers. A custom build makes sense when you need deep integration with your own systems, specific guardrails, multiple languages with strict quality control, or control over the model and data. Mraviko builds custom support assistants in that second category; for many smaller teams, the built-in option is the right first step.
Launch checklist
- Ticket analysis done; scope agreed
- Knowledge base cleaned and owned
- RAG pipeline returns relevant passages for test questions
- System prompt plus code-level guardrails in place
- Customer authentication before any account lookup
- Human handoff with full context
- Test set passes review; soft launch planned
- Dashboards and alerts for the metrics above
Final thought
Building an AI chatbot for customer support is less about the model and more about the work around it: clear scope, accurate content, grounded answers, firm guardrails and a graceful exit to a human. Get those right and the chatbot handles the repetitive questions well, while your team spends its time on the conversations that actually need people.
