RAG Assistants for Ugandan Businesses: Building Your First AI Helper

Every Ugandan business leader we talk to in 2025 asks the same question: 'How can AI help my business?' The honest answer, until recently, was: 'Not much — generic ChatGPT doesn't know your policies, your products, or your customers.' That changed with RAG — Retrieval-Augmented Generation. RAG lets you build AI assistants that actually understand your business, without training a model from scratch. This guide explains how.

What is RAG?

RAG (Retrieval-Augmented Generation) is a technique that combines a large language model (LLM) like GPT-4 or Claude with your own documents. Instead of relying on what the model learned during training, RAG retrieves relevant documents from your knowledge base and feeds them to the model at inference time. The model then generates a response grounded in your actual data.

Why RAG beats fine-tuning

Until 2023, the standard way to make an AI 'know' your business was fine-tuning — retraining a model on your data. Fine-tuning has three problems:

• Expensive — fine-tuning GPT-3.5 costs $0.008 per 1K tokens, and you need thousands of examples
• Static — every time your data changes (new product, new policy), you have to re-fine-tune
• Opaque — you can't see why the model gave a particular answer, or what document it's drawing from

RAG solves all three. It's cheap (you pay only for inference, not training), dynamic (you can update your documents anytime), and transparent (you can see exactly which documents were retrieved for each answer). For 95% of business AI use cases, RAG is the right choice.

What can RAG assistants do for Ugandan businesses?

We've built RAG assistants for clients across industries. Here are the use cases that deliver the most value:

• Internal knowledge base — employees ask 'What's our refund policy?' or 'How do I configure the VPN?' and get instant, accurate answers grounded in your actual documents
• Customer support — customers ask 'How do I reset my password?' or 'What are your opening hours?' and the assistant answers from your help docs, escalating to a human only when needed
• Sales enablement — sales reps ask 'What's the pricing for the enterprise tier?' or 'How does our product compare to competitor X?' and get instant, accurate answers
• Legal/policy Q&A — staff ask 'Can I expense this?' or 'What's our data retention policy?' and get answers from your actual policies
• Product documentation — developers ask 'How do I authenticate to the API?' and get code samples from your docs

How to build a RAG assistant

A production RAG system has four components:

1. Document ingestion

First, you load your documents (PDFs, Word, HTML, Markdown, Google Docs) and split them into chunks. Chunk size matters — too small and you lose context, too large and retrieval is imprecise. We typically use 500-1000 tokens per chunk with 100-token overlap.

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=100,
    separators=['\n\n', '\n', '. ', ' '],
)
chunks = splitter.split_text(document_text)

2. Embedding and indexing

Each chunk is converted to a vector embedding (a list of numbers representing the chunk's meaning) using an embedding model (OpenAI text-embedding-3-small, Cohere, or open-source alternatives like BGE). These vectors are stored in a vector database.

• Pinecone — managed, easy to use, good free tier
• Weaviate — open-source, self-hostable, GraphQL API
• pgvector — PostgreSQL extension, runs in your existing Postgres database (our default choice)
• Qdrant — open-source, fast, Rust-based

3. Retrieval

When a user asks a question, you embed their question using the same embedding model, then search the vector database for the most similar chunks. You typically retrieve the top 3-5 most relevant chunks.

from pgvector.psycopg import register_vector

# Embed the question
question_embedding = openai.embeddings.create(
    input=user_question,
    model='text-embedding-3-small'
).data[0].embedding

# Retrieve top 5 most similar chunks
cur.execute('''
    SELECT content, metadata, embedding <=> %s AS distance
    FROM documents
    ORDER BY distance
    LIMIT 5
''', (question_embedding,))

4. Generation

Finally, you send the retrieved chunks + the user's question to an LLM (GPT-4, Claude, Llama) and ask it to answer the question using only the provided context. This is where the magic happens — the model synthesises a natural-language answer grounded in your actual data.

prompt = f'''You are a helpful assistant for {company_name}.
Use the following context to answer the question.
If the context doesn't contain the answer, say 'I don't know'.

Context:
{retrieved_chunks}

Question: {user_question}

Answer:''' }

Production considerations

Building a RAG demo is easy. Building a production RAG system that's accurate, fast and cheap is hard. Here are the things that matter:

• Chunking strategy — experiment with chunk size and overlap. Smaller chunks = more precise retrieval but less context.
• Re-ranking — after retrieval, use a cross-encoder to re-rank chunks by relevance. This improves answer quality by 20-30%.
• Citations — always show which document each answer came from. This builds user trust and enables verification.
• Fallback — if the assistant can't find a relevant document, it should say 'I don't know' and escalate to a human. Hallucinated answers destroy trust.
• Caching — cache common questions to reduce LLM costs. We've seen 40-60% cache hit rates on production assistants.
• Monitoring — log every question, retrieved chunks, and answer. Review these weekly to identify gaps in your knowledge base.

Costs in 2025

RAG has gotten dramatically cheaper in 2025. Here's what a typical production assistant costs per month:

• Embeddings — ~$5/month for 100K chunks (using OpenAI text-embedding-3-small at $0.02/1M tokens)
• Vector database — $0 if you use pgvector in your existing Postgres, $70+/month for managed (Pinecone)
• LLM inference — $50-500/month depending on volume (using GPT-4o-mini at $0.15/1M input tokens)
• Hosting — $20-100/month for the application server

Total: $75-625/month for a production RAG assistant. For most Ugandan businesses, this pays for itself within a month in saved staff time.

Want to build one?

We build RAG assistants for Ugandan businesses — from scoping to production in 4-6 weeks. Book an AI discovery call and we'll help you identify the highest-ROI use case for your business.