Skip to content

Latest commit

 

History

History
149 lines (119 loc) · 6 KB

architecture.md

File metadata and controls

149 lines (119 loc) · 6 KB

Architecture

This document explains how the pieces fit together and why they are shaped the way they are.

Module map

src/context_guard/
├── settings.py          Typed configuration from environment and .env
├── errors.py            Exception hierarchy
├── context_loader.py    Profile loading: files or CG_PROFILE_INLINE
├── prompting.py         Hardened system prompt, boundaries, canary
├── registry.py          Curated free-tier model catalog
├── client.py            google-genai wrapper: retries, rate limit, timeouts
├── engine.py            The five-layer guard pipeline
├── cli.py               Typer commands: chat, ask, doctor, models
├── webapi.py            FastAPI layer for the web UI
└── security/
    ├── sanitizer.py     Layer 1: Unicode normalization and size limits
    ├── injection.py     Layer 2: injection signature scanner
    ├── output_guard.py  Layer 5: leak detection on responses
    └── rate_limiter.py  Outbound token bucket, inbound sliding window

api/index.py             Vercel entrypoint for the Python service
web/                     The Next.js web UI service (App Router, Tailwind)
vercel.json              Vercel Services definitions and public routing

Request flow

sequenceDiagram
    participant U as User
    participant E as ContextGuardEngine
    participant S as Sanitizer
    participant I as Injection scanner
    participant C as GeminiClient
    participant G as Gemini API

    U->>E: ask(question)
    E->>S: sanitize_input
    S-->>E: cleaned text or rejection
    E->>I: scan(cleaned)
    I-->>E: score and matched signals
    alt score >= threshold
        E-->>U: blocked, zero tokens spent
    else clean enough
        opt scope pre-check enabled
            E->>C: classify with lite model
            C->>G: generate_content
            G-->>C: IN_SCOPE or OUT_OF_SCOPE
        end
        E->>C: generate with hardened prompt
        C->>G: generate_content (rate limited, retried)
        G-->>C: response text
        C-->>E: text
        E->>E: output guard review
        E-->>U: answer or refusal
    end
Loading

Design decisions

Why generate_content instead of the newer Interactions API

The Gemini docs now feature an Interactions API alongside the classic generateContent endpoint. This project deliberately uses client.models.generate_content because it is the mature, fully supported surface with stable semantics for system instructions, multi-turn contents, safety settings, and token limits, and it works uniformly across every model in the registry. The client wrapper isolates this choice in one method, so migrating later means changing one file.

Why the scope pre-check fails open

If the lite classifier model is unavailable (quota exhausted, network issue), the engine logs a warning and proceeds. The hardened system prompt still enforces scope at the main model, so failing open trades a little strictness for a lot of availability. Failing closed would make the whole tool unusable whenever one cheap model has a bad day.

Why blocked exchanges never enter history

Only successful, guard-approved exchanges are stored in the conversation history. A blocked injection attempt that entered history would give the attacker a second chance to influence later turns.

Why the boundary and canary rotate per session

secrets.token_hex generates the boundary marker and canary token. An attacker cannot pre-craft input that fakes the closing boundary because the marker is unknown to them, and any response containing the canary is proof of prompt leakage. /reset in chat rotates both.

Why the SDK import is lazy

client.py imports google-genai inside the constructor. The entire guard pipeline (sanitizer, scanner, prompting, output guard, engine) is therefore unit-testable with a fake client and no SDK, which keeps the test suite fast and network-free.

Why there is a client-side rate limiter

The free tier enforces per-model requests-per-minute and requests-per-day quotas server-side. Hitting those limits produces 429 noise and wasted retries. A token bucket below the ceiling keeps usage polite and predictable, and it is the single choke point if this engine is ever embedded somewhere bigger.

The web layer

The web UI is a Next.js App Router application served by Vercel, talking to context_guard.webapi, a FastAPI app deployed as one Python serverless function. The important shape decisions:

Stateless by design

Serverless instances come and go, so the server holds no session state. The browser stores the conversation in localStorage and replays it with every request. The replay is untrusted input: ContextGuardEngine.load_history runs every replayed message through the sanitizer and the injection scanner, and silently drops any pair that fails, so a hand-crafted "history" cannot smuggle instructions past the guards.

Shared client, fresh engine

The Gemini client (with its outbound token bucket) is cached per warm instance so connections and rate accounting are reused. The engine itself is constructed per request, which guarantees two visitors can never share conversation state.

Why responses do not stream

Layer 5 reviews the complete response before anyone sees it. Streaming would show tokens before the guard could veto them, so the API returns complete, reviewed answers, and the UI renders that Markdown and fades it in. Rendering happens with react-markdown and no raw-HTML plugin, so any HTML a model emits is escaped to inert text rather than executed. Security wins over spectacle here.

Why the inbound limiter is in memory

The per-IP sliding window lives in function memory, which on serverless means per warm instance. That is a deliberate trade: it blocks the casual abuse that actually happens, requires no external store, and platform-level firewall rules remain the right tool for hard guarantees. The limiter is a single class (SlidingWindowLimiter) that can be swapped for a Redis-backed implementation without touching any route.