Retrieval-Augmented Generation (RAG) is the industry standard pattern for powering LLMs with external, dynamic, or private knowledge bases without the need for expensive model fine-tuning.
This guide walks you through building a modular, production-ready RAG application from scratch using **TypeScript**, **LangChain**, and **ChromaDB**.
Conceptual Architecture
[Document Source] ──> [Loader & Splitter] ──> [Vector Store]
│
[User Query] ──────> [Embedding model] ────────────┤
▼
[Augmented Prompt] <── [LLM Generator] <── [Top-K Context]---
Step 1: Chunking & Embedding Strategies
To search documents efficiently, we first divide large texts into smaller paragraphs ("chunks") and transform them into high-dimensional semantic vectors using an embedding model.
Here is how to load and chunk text files using LangChain:
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { Document } from "@langchain/core/documents";
// Define text splitter with overlaps to maintain context boundaries
const splitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
chunkOverlap: 200,
});
const docs = [
new Document({
pageContent: "AI agent architectures rely on planning, memory, and tool utilization modules...",
metadata: { source: "agent_guide.txt" }
})
];
const chunks = await splitter.splitDocuments(docs);
console.log(`Generated ${chunks.length} semantic document chunks.`);---
Step 2: Storing in a Vector Database
Once chunked, we ingest the documents into a vector database (e.g. Chroma, Pinecone, or pgvector).
import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { OpenAIEmbeddings } from "@langchain/openai";
// Instantiate embedding provider
const embeddings = new OpenAIEmbeddings({
model: "text-embedding-3-small",
});
// Initialize vector store with loaded documents
const vectorStore = await MemoryVectorStore.fromDocuments(chunks, embeddings);---
Step 3: Prompt Augmentation & Generation
Now, when a user asks a query, we retrieve the top-K matching chunks, format them into the prompt layout, and generate the answer.
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";
// 1. Setup Retriever
const retriever = vectorStore.asRetriever({ k: 3 });
const contextDocs = await retriever.invoke("What are AI agent architectures?");
const contextText = contextDocs.map(d => d.pageContent).join("\n\n");
// 2. Build Prompt Template
const prompt = ChatPromptTemplate.fromMessages([
["system", "You are an assistant. Answer using ONLY this context:\n\n{context}"],
["human", "{question}"]
]);
// 3. Generate response
const model = new ChatOpenAI({ modelName: "gpt-4o-mini" });
const chain = prompt.pipe(model);
const response = await chain.invoke({
context: contextText,
question: "What are AI agent architectures?"
});
console.log(response.content);Production Readiness Checklist
- **Hybrid Search**: Combine vector similarity (dense) with keyword matching (BM25) to capture both semantic meaning and exact terms.
- **Re-ranking**: Run retrieved documents through a Cross-Encoder model (like Cohere ReRank) to refine the top-K order before prompt assembly.