A full-stack Retrieval-Augmented Generation (RAG) system. Upload PDFs, text files, code, or web URLs — then chat with your documents using a streaming LLM backed by semantic search.
frontend (React + Vite) → backend (FastAPI) → ChromaDB (vector store)
→ Groq API (LLM, via OpenAI compat)
→ Sentence Transformers (embeddings)
Ingestion pipeline: File/URL → Loader → Chunker → Embedder → ChromaDB
Query pipeline: User query → Embed → Retrieve → Rerank → LLM stream → Response
- Python 3.11+
- Node.js 22+ with pnpm
- uv (Python package manager)
- A Groq API key (free tier available)
# Backend env
cp backend/.env.example backend/.env
# Edit backend/.env and set your GROQ_API_KEY# Frontend env (only needed if backend runs on a non-default port)
cp frontend/.env.example frontend/.envcd backend
uv sync # install dependencies
make dev # runs uvicorn with --reload on port 8000Or manually:
uv run uvicorn app.main:app --reload --host 0.0.0.0 --port 8000cd frontend
pnpm install
pnpm dev # Vite dev server on port 5173Open http://localhost:5173.
The Vite dev server proxies /api/v1 to http://localhost:8000, so no CORS issues during development.
# Copy and fill in your API key first
cp backend/.env.example backend/.env
docker compose up --build- Frontend: http://localhost:5173
- Backend: http://localhost:8000
- ChromaDB data persists in the
chroma_dataDocker volume.
| Variable | Default | Description |
|---|---|---|
GROQ_API_KEY |
(required) | Groq API key |
GROQ_MODEL |
llama-3.1-8b-instant |
LLM model name |
EMBEDDING_MODEL |
all-MiniLM-L6-v2 |
Sentence Transformers model |
CHROMA_PERSIST_DIR |
./data/chroma_db |
ChromaDB storage path |
CHROMA_COLLECTION_NAME |
rag_documents |
ChromaDB collection name |
CHUNK_SIZE |
512 |
Token size per chunk |
CHUNK_OVERLAP |
64 |
Overlap between chunks |
TOP_K_RETRIEVAL |
10 |
Chunks retrieved before reranking |
TOP_K_AFTER_RERANK |
4 |
Chunks passed to the LLM after reranking |
RERANKER_MODEL |
cross-encoder/ms-marco-MiniLM-L-6-v2 |
Cross-encoder reranker model |
MAX_CONVERSATION_TURNS |
20 |
Number of conversation turns kept in memory |
UPLOAD_MAX_SIZE_MB |
50 |
Maximum upload file size |
| Variable | Default | Description |
|---|---|---|
VITE_API_BASE_URL |
/api/v1 |
Backend API base URL (override for non-proxied deployments) |
Go to the Upload page (/upload):
- Drag & drop or click to upload files. Supported formats:
- PDF (
.pdf) - Word documents (
.docx) - Plain text / Markdown (
.txt,.md) - Code (
.py,.js,.ts,.tsx,.jsx,.go,.rs,.java,.cpp,.c,.cs,.rb,.php,.sh,.sql,.kt,.swift)
- PDF (
- Paste a URL to ingest a web page directly.
Go to the Chat page (/):
- Type a question in the input field.
- The assistant streams an answer backed by your uploaded documents.
- Source excerpts (with page numbers for PDFs) appear in the side panel.
- Each browser session gets its own conversation history (up to
MAX_CONVERSATION_TURNS). - Click New session to reset the conversation.
Go to the Documents page (/documents):
- View all ingested documents with chunk counts and source types.
- Delete a document to remove it and all its chunks from the vector store.
Base URL: http://localhost:8000/api/v1
GET /health
Returns { "status": "ok", "document_count": <n> }.
POST /ingest/file
Content-Type: multipart/form-data
Body: file=<binary>
POST /ingest/url
Content-Type: application/json
Body: { "url": "https://example.com/article" }
Both return a DocumentSummary:
{
"doc_id": "uuid",
"filename": "report.pdf",
"source_type": "pdf",
"chunk_count": 42,
"status": "ready",
"created_at": "2024-01-01T00:00:00Z",
"url": null
}GET /documents → list all DocumentSummary objects
DELETE /documents/{doc_id} → remove document and its chunks
POST /chat/stream
Content-Type: application/json
Body: {
"query": "What does the document say about X?",
"session_id": "uuid",
"filter_doc_ids": ["doc_id_1"] // optional: limit to specific docs
}
Returns a Server-Sent Events stream. Each event is one of:
| Event | Payload |
|---|---|
delta |
{ "text": "..." } — streamed LLM token |
sources |
[{ "chunk_id", "doc_id", "filename", "score", "excerpt", "page" }, ...] |
done |
{} |
error |
{ "message": "..." } |
DELETE /chat/session/{session_id} → clear conversation history
POST /search
Content-Type: application/json
Body: { "query": "...", "top_k": 5 }
Returns { "results": [RetrievedChunk, ...] }.
smart-rag/
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI app, CORS, router mount
│ │ ├── config.py # Pydantic settings (reads .env)
│ │ ├── dependencies.py # lru_cache singleton services
│ │ ├── api/
│ │ │ ├── router.py # aggregates all routes under /api/v1
│ │ │ ├── ingest.py # /ingest/file, /ingest/url
│ │ │ ├── documents.py # /documents
│ │ │ ├── chat.py # /chat/stream, /chat/session
│ │ │ └── search.py # /search
│ │ ├── services/
│ │ │ ├── ingestion_service.py # file/url → chunks → vectors
│ │ │ ├── retrieval_service.py # embed query → retrieve → rerank
│ │ │ ├── vector_store.py # ChromaDB wrapper
│ │ │ ├── embedding_service.py # Sentence Transformers
│ │ │ ├── llm_service.py # Groq streaming via OpenAI SDK
│ │ │ ├── chunking_service.py # LangChain text/code splitter
│ │ │ ├── reranker_service.py # Cross-encoder reranking
│ │ │ └── conversation_service.py # Per-session chat history
│ │ ├── loaders/
│ │ │ ├── pdf_loader.py # PyMuPDF
│ │ │ ├── url_loader.py # httpx + trafilatura
│ │ │ ├── text_loader.py # plain text / markdown
│ │ │ └── code_loader.py # source code with language detection
│ │ ├── models/ # Pydantic data models
│ │ └── utils/
│ │ └── file_utils.py # MIME/extension → loader key detection
│ ├── Makefile # dev, test, lint, format targets
│ ├── pyproject.toml
│ └── Dockerfile
├── frontend/
│ ├── src/
│ │ ├── api/
│ │ │ ├── client.ts # axios instance (base URL, timeout, error interceptor)
│ │ │ ├── chat.ts # SSE streaming, clearSession
│ │ │ ├── ingest.ts # ingestFile, ingestUrl
│ │ │ └── documents.ts # listDocuments, deleteDocument
│ │ ├── components/
│ │ │ ├── chat/ # ChatWindow, MessageList, MessageBubble, ChatInput, SourcePanel
│ │ │ ├── upload/ # DropZone
│ │ │ ├── documents/ # DocumentList
│ │ │ └── common/ # Layout, Sidebar, Spinner
│ │ ├── hooks/
│ │ │ ├── useChat.ts # streaming message logic
│ │ │ ├── useDocuments.ts # document list + delete
│ │ │ └── useIngestion.ts # upload progress and error state
│ │ ├── store/
│ │ │ ├── chatStore.ts # Zustand: messages, streaming flag
│ │ │ └── documentStore.ts # Zustand: documents array
│ │ └── types/index.ts # shared TypeScript interfaces
│ ├── vite.config.ts # dev server port 5173, proxy /api/v1 → :8000
│ ├── .npmrc # points to public npm registry
│ └── Dockerfile # multi-stage: Node build → nginx
├── docker-compose.yml
├── deploy.sh
└── k8s/ # Kubernetes manifests (namespace, deployment, pvc, service)
make dev # uvicorn with hot reload
make test # pytest with coverage
make lint # ruff check
make format # ruff formatpnpm dev # Vite dev server
pnpm build # production build
pnpm preview # preview production build locallykubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/secret.yaml # add GROQ_API_KEY first
kubectl apply -f k8s/configmap.yaml
kubectl apply -f k8s/pvc.yaml
kubectl apply -f k8s/backend.yaml
kubectl apply -f k8s/frontend.yamlFrontend is exposed as a NodePort service on port 30080.
Backend readiness is probed at GET /api/v1/health.