This project treats every user message as hostile until proven otherwise. This document describes the threat model, the five defense layers, and the residual risks that remain.
Assets to protect:
- The scope constraint: the assistant must only discuss the profile owner.
- The system prompt and its security tokens.
- The API key.
- The free-tier quota (an attacker should not be able to burn it cheaply).
Adversary capabilities: full control over the text of every question, including Unicode tricks, encodings, multi-turn setups, and social engineering of the model. The adversary cannot see the server side, the environment, or the source of randomness.
| Layer | Module | Defends against |
|---|---|---|
| 1. Sanitizer | security/sanitizer.py |
Invisible characters, full-width and compatibility Unicode disguises, control characters, oversized inputs |
| 2. Injection scanner | security/injection.py |
Known injection phrasing, role markers, jailbreak keywords, encoded payloads, delimiter mimicry |
| 3. Scope pre-check | engine.py |
Off-topic questions, using a second model as an independent judge |
| 4. Hardened prompt | prompting.py |
Instruction override, persona hijack, prompt extraction |
| 5. Output guard | security/output_guard.py |
Successful prompt leaks, canary exposure, empty or blocked responses |
NFKC normalization folds disguised characters into canonical forms before pattern matching, so an attack written in full-width characters looks like plain ASCII to the scanner. Zero-width characters, bidirectional control marks, and other invisible code points are stripped, closing the hidden-text channel. Input length is capped to bound both cost and attack surface.
A weighted signature set scores each message: instruction overrides, prompt extraction attempts, persona hijacks, chat role markers, jailbreak keywords, authority claims, base64 and hex blobs, and attempts to mimic the project's own delimiters. At or above the threshold (default 3) the message is blocked locally, which costs the attacker a try and costs you zero tokens. The signatures are intentionally conservative so normal questions do not trip them.
An independent, cheap model receives only a preview of the profile and the question, and answers IN_SCOPE or OUT_OF_SCOPE. Because this classifier has its own instructions and no conversation history, an injection crafted for the main conversation rarely transfers to it. The check fails open by design, see architecture.md.
The system instruction pins the model to the profile context and defines an
exact refusal sentence. The user message is spotlighted: wrapped in boundary
markers built from secrets.token_hex, with an explicit rule that everything
inside the markers is data, never instructions. The prompt also carries a
canary token the model is told to never output. Low temperature (default 0.2)
reduces creative drift away from the rules.
The response is scanned for the canary, the rule sentinel, the boundary marker, and the context delimiters. Any hit means the model was manipulated into leaking its instructions, and the response is replaced with the standard refusal. Empty responses (typically safety-filter blocks) are also converted to refusals instead of confusing blank output.
- The API key lives in the environment only, is typed as
SecretStr, and is masked in diagnostics output. .gitignoreexcludes.envand personal profile files by default.- A client-side token bucket keeps request rates under free-tier ceilings.
- Retries are bounded with jittered exponential backoff, so failures cannot turn into request storms.
- Explicit Gemini safety settings block harmful content categories at medium-and-above thresholds.
- CI runs bandit, pip-audit, and gitleaks on every push, and the bandit trojan-source check guards against bidirectional-character attacks on this project's own source code.
- Blocked exchanges never enter conversation history.
The public web UI adds an internet-facing surface with its own controls:
| Threat | Control |
|---|---|
| Poisoned conversation replay | Every replayed message passes the sanitizer and injection scanner server-side; failing pairs are dropped before they reach the model |
| Fabricated assistant turns in the replay | Assistant messages in the replay are scanned with the same signatures as questions |
| Request flooding and quota burning | Per-IP sliding window limiter, then the outbound token bucket, then Gemini's own quotas. Injection attempts are blocked before any token is spent |
| Arbitrary model invocation | The model field is validated against the curated free-tier allowlist |
| Information leakage through errors | Failures return a generic message; details go to server logs only. API docs endpoints are disabled |
| Clickjacking, script injection, sniffing | Strict security headers on every page: a same-origin Content Security Policy, X-Frame-Options DENY, nosniff, HSTS, and a locked-down Permissions-Policy |
| Response caching of personal answers | Every API response carries Cache-Control: no-store |
| Streaming leaks | Responses never stream; the output guard reviews the complete text before the browser receives anything. The client then renders the reviewed Markdown |
| Rendered-answer XSS | Answers are rendered with react-markdown and no raw-HTML plugin, so HTML in a model response is escaped, not executed. Links are forced to open with rel="noopener noreferrer nofollow" |
- A novel, well-crafted injection can still get an off-topic sentence out of the model. No known technique reduces this risk to zero. The layers here make the cheap attacks free to block and the expensive attacks detectable when they matter most (prompt leakage).
- The heuristic scanner can produce false positives on unusual but legitimate
questions. Tune
CG_INJECTION_THRESHOLDif your use case trips it. - The profile context itself is sent to Google's API with every request. Only put information there that you are comfortable sharing with the API provider and with anyone who chats with the assistant.
- Free-tier quotas are enforced per project by Google. This tool limits its own rate but cannot stop other consumers of the same key.