PROGRAM ZERO 18-month live AI, LLM & full-stack programme · ₹89,999 or ₹7,500/month · Starts 9 January 2027
Explore Program Zero →
Careers Ninza — business and startup leadership training
JOIN ZERO NINZA KIDS
JOIN ZERO NINZA KIDS
See all programs → Free live masterclass → Guides & articles → Newsletter →
Careers Ninza
AI & AUTOMATION 6 min read · Updated 20 October 2026

Build a chatbot that answers from your own documents, without coding: RAG explained

Retrieval-augmented generation (RAG) lets an AI answer from your own policies, manuals and FAQs instead of guessing. This guide explains chunking, embeddings, vector stores and citations, then builds a policy assistant in n8n step by step.

CN
Careers Ninza AI faculty
Careers Ninza · Kolkata, India
Learn this live in our No-Code Agentic AI Development course

Retrieval-augmented generation (RAG) means that before an AI answers, the system searches your own documents for the most relevant passages and gives them to the model with the question. The model answers from those passages and can cite them. It is the standard way to build assistants that know your policies, products or procedures without retraining a model.

Businesses ask for this more than any other AI project: "Can we have a chatbot that answers from our HR policy, product manuals or course FAQs?" It is module six of No-Code Agentic AI Development, and students build one on their own documents. For the deeper, code-level view of retrieval, see our LLM course guide.

Why not just paste documents into ChatGPT or Claude?

For a few documents and personal use, that works well, and tools such as Claude Projects do exactly this. RAG becomes necessary when documents are many or large, change often, must be shared with many users through a website or WhatsApp, need access control, or when answers must cite sources reliably. RAG keeps the knowledge outside the model and searchable.

How does RAG work, step by step?

StageWhat happensPlain-language version
LoadDocuments are read from Drive, PDFs, web pages or a databaseCollect the books
ChunkText is split into passages of a few hundred words with some overlapCut them into index cards
EmbedEach chunk becomes a vector, a list of numbers representing its meaningGive each card a meaning code
StoreVectors are saved in a vector storeFile the cards
RetrieveThe question is embedded and the closest chunks are foundFind the most relevant cards
GenerateThe model answers using only those chunks, with citationsAnswer from the cards and say which ones

What do you need to build it in n8n?

—Documents in an accessible place (Google Drive folder, a website, a database).
—An embedding model (for example from OpenAI, Google, Cohere or Voyage AI, or a local model) and a chat model for answers.
—A vector store: Pinecone, Qdrant, Supabase (pgvector) or n8n’s simple in-memory store for testing.
—Two workflows: one to ingest documents, one to answer questions.

Workflow 1: ingesting documents

1Trigger on a schedule or when a file changes in the Drive folder.
2Download the file and extract text (PDF, DOCX, Google Docs).
3Split with a text splitter: start around 500 to 800 tokens per chunk with 10% to 20% overlap, and keep headings with their text.
4Add metadata: document name, section, last updated date, audience (for example "employees" or "customers").
5Embed and upsert into the vector store, replacing old versions of the same document.

Workflow 2: answering questions

1Trigger from a chat widget, WhatsApp or Slack.
2Retrieve the top few chunks for the question, filtered by metadata such as audience.
3Send them to the chat model with a strict prompt: "Answer only from the context. If the answer is not in the context, say you do not know and suggest contacting HR. Cite the document and section for each fact."
4Return the answer with citations and log the question, retrieved chunks and answer.

The most important line in a RAG prompt is permission to say "I don’t know". Without it, models fill gaps with plausible guesses, which is the main cause of wrong answers in document chatbots.

A worked example: an HR policy assistant

A Gurugram company with 300 employees uploads its leave policy, travel policy, reimbursement rules and holiday list. Employees ask on Slack: "How many casual leaves do I get?", "Is Uber reimbursed for client visits in Noida?", "Is Chhath a holiday for the Patna office?" The assistant answers with the relevant clause and document name, and says "I don’t know, please ask HR" when the policy is silent. HR reviews the log weekly to find missing policies and confusing wording.

How do you test and improve answer quality?

1Write 40 to 60 real questions with expected answers and source documents.
2Check retrieval first: did the right chunks come back? If not, adjust chunk size, overlap, metadata filters or the number of chunks retrieved.
3Then check answers: correct, partially correct, wrong, refused correctly.
4Re-run the set after every change to documents, prompts or models.
Illustration: where errors came from in a sample RAG assistant before tuning (40 test questions)
Wrong chunks retrieved9 errors
Answer ignored context3 errors
Document out of date4 errors
Correct24 correct

Source: Careers Ninza classroom project; illustrative. Most errors in RAG come from retrieval and content, not the model.

What are the limits of RAG?

RAG is only as good as the documents: outdated, contradictory or badly written policies produce poor answers. It struggles with questions that need calculations across many documents or tables, and with scanned images unless text is extracted well. Personal data in documents must be handled under India’s DPDP Act, with access control so users see only what they are allowed to.

How do you prepare documents so RAG works well?

The quality of answers depends more on document preparation than on the model. Before ingesting, we ask clients to:

1Remove duplicates and old versions so the assistant does not quote last year’s policy.
2Use clear headings so chunks carry their context ("Leave policy > Casual leave > Eligibility").
3Convert scanned PDFs to text with OCR and check the result, especially tables.
4Write missing answers: if employees often ask something the policy does not cover, add an FAQ.
5Add owners and dates to every document so someone is responsible for updates.

How does metadata filtering make answers safer?

Metadata lets you restrict which chunks can be retrieved. Tag chunks with audience (employees, managers, customers), location (Kolkata office, Pune plant), and confidentiality. Then filter retrieval by the user’s role and location, so a plant worker in Pune gets the Pune shift policy and never sees management-only documents. This is also how you meet access-control expectations under data protection rules.

When should you use long context instead of RAG?

Modern models accept very long inputs, so for a single handbook or a few contracts you can sometimes send the whole document with the question. This is simpler and avoids retrieval errors, but it costs more per question and does not scale to hundreds of documents. A practical rule: long context for a few documents used occasionally, RAG for many documents, many users or frequent questions. Claude Projects and similar features are a good middle ground for small teams.

Which use cases work best for Indian businesses?

—HR and policy assistants for employees.
—Product and spare-part assistants for dealers and service engineers.
—Course and admission FAQ assistants for institutes.
—SOP assistants for factories, hospitals and hotels.
—Tender and contract search for project teams.

Our No-Code Agentic AI Development batches run live online, so the same class reaches learners and business owners in metros such as Bengaluru, Pune, Kolkata, Mumbai and Delhi NCR, growing cities such as Jaipur, Kanpur, Madurai, Raipur and Dehradun, and smaller industrial towns such as Panipat, Firozabad, Karur, Durgapur and Dibrugarh, at the same fee. Each city page explains how local businesses use AI knowledge assistants, for example in Jaipur, Panipat, Madurai.

Learn this live: No-Code Agentic AI Development

Build a RAG assistant on real documents with retrieval tuning, citations, evaluation and access control. 40 live hours, ₹34,999 paid in full or ₹40,000 as No-Cost EMI of ₹20,000 × 2.

LONG-TERM PROGRAM · LIVE ONLINE
No-Code Agentic AI Development

Ship AI agents that do real work — without writing production code.

₹34,999 paid in full · EMI ₹20,000 × 240 hoursLive, mentor-ledCertificate on completion

Frequently asked questions

What is RAG in AI?+

Retrieval-augmented generation is a method where the system first searches your documents for relevant passages and then gives them to a language model to answer from, often with citations. It lets AI answer from private, current information without retraining the model.

Can I build a RAG chatbot without coding?+

Yes. Tools such as n8n provide document loaders, text splitters, embedding nodes, vector store nodes and AI chat nodes that you connect visually to build ingestion and question-answering workflows.

What is a vector database?+

A vector database stores embeddings, numerical representations of text meaning, and finds the passages most similar in meaning to a question. Examples include Pinecone, Qdrant and PostgreSQL with pgvector.

How do I stop a RAG chatbot from making things up?+

Instruct it to answer only from retrieved context and to say it does not know when the answer is missing, require citations, tune retrieval so the right passages are found, and test with a set of real questions.

What chunk size should I use for RAG?+

Start around 500 to 800 tokens with 10% to 20% overlap, keeping headings with their content, then adjust based on test results. Policies and FAQs often work well with section-based chunks.

Where can I learn RAG in India?+

Module six of Careers Ninza No-Code Agentic AI Development covers retrieval-augmented generation without code. The programme is 40 live hours, ₹34,999 paid in full or ₹40,000 as No-Cost EMI of ₹20,000 × 2.

Related reading

AI & AUTOMATION Rolling Out Claude Across an Indian Company: Pilots, Governance, Training and ROI Buying licences is the easy part of an AI rollout. This guide covers how Indian companies should pilot Claude, set governance and data rules, train people by role, build shared Skills and connectors, control cost and measure real value. 6 min read AI & AUTOMATION AI Automation for HR and Recruitment: Screening, Onboarding and Employee Questions HR teams spend hours on repetitive tasks: sorting CVs, scheduling interviews, onboarding paperwork and answering the same policy questions. This guide shows practical AI automations for Indian HR teams, with fairness and privacy safeguards. 6 min read AI & AUTOMATION Claude for Small Business Owners in India: 20 Practical Uses That Save Time and Money Owners of Indian MSMEs wear every hat: sales, accounts, HR, marketing and operations. This guide lists 20 practical ways Claude helps, from quotations and GST-ready summaries to job descriptions and customer replies, with safety rules and a plan to start. 6 min read

We teach this, live

Every article here comes from something we teach. Sit in on a free masterclass and judge the mentors yourself.