This document explains how the pieces fit together and why they are shaped the way they are.
src/context_guard/
├── settings.py Typed configuration from environment and .env
├── errors.py Exception hierarchy
├── context_loader.py Profile loading: files or CG_PROFILE_INLINE
├── prompting.py Hardened system prompt, boundaries, canary
├── registry.py Curated free-tier model catalog
├── client.py google-genai wrapper: retries, rate limit, timeouts
├── engine.py The five-layer guard pipeline
├── cli.py Typer commands: chat, ask, doctor, models
├── webapi.py FastAPI layer for the web UI
└── security/
├── sanitizer.py Layer 1: Unicode normalization and size limits
├── injection.py Layer 2: injection signature scanner
├── output_guard.py Layer 5: leak detection on responses
└── rate_limiter.py Outbound token bucket, inbound sliding window
api/index.py Vercel entrypoint for the Python service
web/ The Next.js web UI service (App Router, Tailwind)
vercel.json Vercel Services definitions and public routing
sequenceDiagram
participant U as User
participant E as ContextGuardEngine
participant S as Sanitizer
participant I as Injection scanner
participant C as GeminiClient
participant G as Gemini API
U->>E: ask(question)
E->>S: sanitize_input
S-->>E: cleaned text or rejection
E->>I: scan(cleaned)
I-->>E: score and matched signals
alt score >= threshold
E-->>U: blocked, zero tokens spent
else clean enough
opt scope pre-check enabled
E->>C: classify with lite model
C->>G: generate_content
G-->>C: IN_SCOPE or OUT_OF_SCOPE
end
E->>C: generate with hardened prompt
C->>G: generate_content (rate limited, retried)
G-->>C: response text
C-->>E: text
E->>E: output guard review
E-->>U: answer or refusal
end
The Gemini docs now feature an Interactions API alongside the classic
generateContent endpoint. This project deliberately uses
client.models.generate_content because it is the mature, fully supported
surface with stable semantics for system instructions, multi-turn contents,
safety settings, and token limits, and it works uniformly across every model
in the registry. The client wrapper isolates this choice in one method, so
migrating later means changing one file.
If the lite classifier model is unavailable (quota exhausted, network issue), the engine logs a warning and proceeds. The hardened system prompt still enforces scope at the main model, so failing open trades a little strictness for a lot of availability. Failing closed would make the whole tool unusable whenever one cheap model has a bad day.
Only successful, guard-approved exchanges are stored in the conversation history. A blocked injection attempt that entered history would give the attacker a second chance to influence later turns.
secrets.token_hex generates the boundary marker and canary token. An
attacker cannot pre-craft input that fakes the closing boundary because the
marker is unknown to them, and any response containing the canary is proof of
prompt leakage. /reset in chat rotates both.
client.py imports google-genai inside the constructor. The entire guard
pipeline (sanitizer, scanner, prompting, output guard, engine) is therefore
unit-testable with a fake client and no SDK, which keeps the test suite fast
and network-free.
The free tier enforces per-model requests-per-minute and requests-per-day quotas server-side. Hitting those limits produces 429 noise and wasted retries. A token bucket below the ceiling keeps usage polite and predictable, and it is the single choke point if this engine is ever embedded somewhere bigger.
The web UI is a Next.js App Router application served by Vercel, talking to
context_guard.webapi, a FastAPI app deployed as one Python serverless
function. The important shape decisions:
Serverless instances come and go, so the server holds no session state. The
browser stores the conversation in localStorage and replays it with every
request. The replay is untrusted input: ContextGuardEngine.load_history
runs every replayed message through the sanitizer and the injection scanner,
and silently drops any pair that fails, so a hand-crafted "history" cannot
smuggle instructions past the guards.
The Gemini client (with its outbound token bucket) is cached per warm instance so connections and rate accounting are reused. The engine itself is constructed per request, which guarantees two visitors can never share conversation state.
Layer 5 reviews the complete response before anyone sees it. Streaming would show tokens before the guard could veto them, so the API returns complete, reviewed answers, and the UI renders that Markdown and fades it in. Rendering happens with react-markdown and no raw-HTML plugin, so any HTML a model emits is escaped to inert text rather than executed. Security wins over spectacle here.
The per-IP sliding window lives in function memory, which on serverless
means per warm instance. That is a deliberate trade: it blocks the casual
abuse that actually happens, requires no external store, and platform-level
firewall rules remain the right tool for hard guarantees. The limiter is a
single class (SlidingWindowLimiter) that can be swapped for a Redis-backed
implementation without touching any route.