Learning RAG Systems My Journey from Beginner to Real Understanding
When I first started exploring AI, I kept seeing words like RAG, embeddings, and vector databases. Everyone seemed to throw these terms around as if everyone magically understood them. The tutorials I found would show a simple code snippet, maybe a quick chatbot demo, and call it a day.
But no one really explained how RAG systems actually work under the hood.
I didn’t want to just use AI. I wanted to actually understand it. And if you’re anything like me, you probably feel the same way.
Why Most AI Articles Felt Empty
Most AI articles today either:
- Show one tiny example without explaining how the pieces connect
- Talk in buzzwords that don’t actually teach you anything real
I realized very quickly that understanding AI (especially RAG) means going deeper. Not just copying code.
So I decided to slow down and ask simple but real questions:
- What is chunking?
- What is an embedding?
- How do vector databases actually work?
- How do all these fit together to make a RAG system?
What I Learned About RAG (Retrieval-Augmented Generation)
Here’s the real flow, now that I understand it:
Knowledge Base
Start with your documents (Markdown files in my case).
const EMBED_MODEL = 'nomic-embed-text';
const CHAT_MODEL = 'mistral';
const DOCS_DIR = './docs';
async function loadDocs() {
const files = await fs.readdir(DOCS_DIR);
const db = [];
for (const file of files) {
const content = await fs.readFile(path.join(DOCS_DIR, file), 'utf-8');
const chunks = chunkText(content, 500);
for (const chunk of chunks) {
const embedding = await getEmbedding(chunk);
db.push({ chunk, embedding });
}
}
console.log(`Embedded ${db.length} text chunks.`)
return db;
}Chunking
Break those documents into small, meaningful parts.
export function chunkText(text: string, size = 500) {
const chunks = [];
for (let i = 0; i < text.length; i += size) {
chunks.push(text.slice(i, i + size));
}
return chunks;
}Embeddings
Use an embedding model to turn each chunk into a vector (a list of numbers that captures the meaning).
async function getEmbedding(text: string) {
const res = await axios.post('http://localhost:11434/api/embeddings', {
model: EMBED_MODEL,
prompt: text,
});
return res.data.embedding;
}How Embedding Works
An embedding model reads a piece of text and converts it into a list of numbers that capture the meaning of that text. In simple terms:
| Text | Vector (Example Numbers) |
|---|---|
| Reset password | [0.24, 0.51, 0.13, …] |
| Recover account password | [0.23, 0.50, 0.15, …] |
| Launch a rocket | [-0.64, 0.19, 0.93, …] |
- Similar meanings (like “Reset password” and “Recover account password”) produce similar vectors.
- Different meanings (like “Launch a rocket”) produce different vectors.
This way, we can find meaning-based matches, not just exact word matches.
Store Embeddings
Save these vectors into a vector database (like ChromaDB or even a local JSON file for now). In my case, I’m storing the embeddings in memory.
Querying
Now, we can search for similar chunks based on the vectors.
async function findRelevantChunks(db: string[], query: string) {
const queryEmbedding = await getEmbedding(query);
const scored = db.map(entry => ({
chunk: entry.chunk,
score: cosineSimilarity(queryEmbedding, entry.embedding),
}));
scored.sort((a, b) => b.score - a.score);
return scored.slice(0, 3).map(x => x.chunk);
}User Question
When a user asks something, embed the question too.
async function askMistral(context: string, question: string) {
const prompt = `Use the following context to answer:\n\n${context.join('\n\n')}\n\nQuestion: ${question}`;
const res = await axios.post('http://localhost:11434/api/generate', {
model: CHAT_MODEL,
prompt: prompt,
stream: false,
});
return res.data.response.trim();
}
const context = await findRelevantChunks(db, question);
const answer = await askMistral(context, question);Similarity Search
Find the most similar chunks from the vector database (JSON file in my case).
function cosineSimilarity(vecA: number[], vecB: number[]): number {
const dot = vecA.reduce((sum, a, idx) => sum + a * vecB[idx], 0);
const normA = Math.sqrt(vecA.reduce((sum, a) => sum + a * a, 0));
const normB = Math.sqrt(vecB.reduce((sum, b) => sum + b * b, 0));
return dot / (normA * normB);
}Important Insight About Search:
In this simple code, we’re manually calculating the cosine similarity between vectors to find the best matches. This is exactly what a real vector database like ChromaDB, Pinecone, or Milvus would do internally. The only difference is:
| What we’re doing manually | What a Vector Database does |
|---|---|
| Calculate cosine similarity between vectors | Same (but optimized, super fast) |
| Sort by score and pick top results | Same |
| Works for small projects | Scales to millions of vectors |
In short:
cosineSimilarity = Brain of vector search.
Manual search = Fine for small apps.
Vector DB = Needed for big, fast production apps.Answer
async function generateAnswer(context: string[], question: string): Promise<string> {
const prompt = `Use the following context to answer:\n${context.join('\n')}\n\nQuestion: ${question}`;
const res = await axios.post('http://localhost:11434/api/generate', {
model: 'mistral',
prompt,
stream: false,
});
return res.data.response.trim();
}In simple words
Markdown Docs --> Chunking --> Embedding --> Save to Vector DB
User asks Question --> Embed Question --> Search DB --> Retrieve Chunks --> Give to Model --> Get AnswerThis flow is now so clear to me that it’s honestly shocking how badly most tutorials explain it.
Why Chunking and Embeddings Matter More Than People Think
What I also realized:
If your chunking is bad (random cuts, huge blocks), your retrieval will be bad.
If your embeddings are low quality (bad models), even good chunks won’t match properly.
Garbage chunks = Garbage retrieval = Garbage answers.
Good chunking + good embeddings = sharp, precise answers, even without a fancy model.
Most of the real “magic” in RAG systems isn’t in the model. It’s in how you prepare and retrieve the right knowledge.
Where I’m Heading Next
Now that I understand the pieces properly, I’m focusing on building:
- Smart document RAG systems using Markdown files as the knowledge base.
- Clean chunking strategies (splitting by sections, logical grouping).
- Embedding everything into a fast vector database like ChromaDB.
I’m also working on a simple tool to help you:
Final Thoughts
If you’re learning AI today, my advice is simple:
Slow down. Understand the basics first.
Don't get distracted by shiny new tools.
Understand what **chunking** and **embedding** really mean.
Build small systems, validate your learning, and THEN scale.(This was a personal journey into RAG systems. I’ll be writing more soon about the small real-world apps I’m building around this knowledge.)
