Skip to content

fix(security)!: stop exporting raw prompts and user ids to telemetry - #774

Open
lahlfors wants to merge 1 commit into
mainfrom
fix/stop-raw-prompt-export
Open

lahlfors wants to merge 1 commit into
mainfrom
fix/stop-raw-prompt-export

Conversation

@lahlfors

Copy link
Copy Markdown
Collaborator

Summary

Stops raw prompts, model responses and end-user identifiers from leaving the process in telemetry. Telemetry goes to the trace store, whose raw-event bucket is retention-locked for 2555 days in every non-staging tier, while the PIA promises a 90-day purge of raw prompts. This fix keeps that promise true by changing what reaches the bucket. Tracked as POAM-2026-225 (#772, stacked on #771).

  1. Removed at every source.
    • The advisor's request span no longer carries the prompt, the response (including cache hits) or enduser.id. It now records cage.principal.verified and the input and output character lengths. thread.id is recorded only when the caller supplies it, because the default session id embeds the user id.
    • Content is replaced by character lengths, never hashes, in all of these:
      • the gateway inference span;
      • the telemetry helper (genai_span, record_completion);
      • the governor spans, which now carry the tool name and violation codes instead of parameters;
      • MCP tracing and the MCP client;
      • the OPA node factory;
      • the NeMo manager, client and actions;
      • the advisor graph nodes and tools.
  2. Fail-closed exporter filter. src/gateway/observability/content_filter.py wraps every exporter: OTLP HTTP and gRPC, console, setup_tracing, and the evaluator's always-on exporter.
    • It strips content- and identity-bearing attributes from spans, events and links.
    • Any span it cannot sanitise is dropped rather than exported. Only counts and exception type names are logged.
  3. Capture flags off. The telemetry client used to force GenAI capture on. It now forces GenAI capture, httpx request and response body capture, and TRACELOOP_TRACE_CONTENT off.
  4. Terraform. Both workload modules set the three capture variables to "false". They also stop capturing the IAP end-user email header. A test fails on any other value.

BREAKING CHANGE: spans no longer carry prompt or response content or end-user identifiers. Indexed as TC-1 in docs/BREAKING_CHANGES.md.

Follow-ups, not in this PR:

  • The OSCAL SSP's Langfuse narrative (compliance/oscal/system-security-plan.yaml, "prompt/response logging") must be restated within 2 business days of merge, per AGENTS.md.
  • Correcting the PIA comes last, under POAM-2026-225.
  • Residual exporters outside the filter, all documented:
    • exception text from record_exception;
    • the compliance bridge's SDK path for remediation text, which is derived from evidence records, not prompts.

Verification

  • uv run pytest -m "local or unit" -n auto -q: 10996 passed, 76 skipped, 0 failed
  • Planted-canary tests: a unique prompt and user id sent through the advisor endpoint, the gateway inference proxy and the telemetry helper appear in no exported span attribute, event or resource. Separate tests assert the advisor request span sets neither even without the filter.
  • Filter tests:
    • listed exact keys and prefixes are dropped from spans and events;
    • allowed keys are kept;
    • a span that raises during sanitisation is dropped, not exported.
  • uv run bandit -r src/ scripts/ -c pyproject.toml -ll -ii: 0 High / 0 Medium
  • scripts/check_import_boundaries.py, check_domain_literals.py, check_telemetry_literals.py, check_vendor_brands.py, check_doc_references.py: clean; terraform fmt -check clean

Spans reach the self-hosted Langfuse over OTLP, and Langfuse writes
every raw ingestion event to a retention-locked bucket (2,555 days in
prod), so a prompt, response or end-user identifier on a span outlives
the 90-day raw-prompt purge that the privacy impact assessment
describes.

Remove content and identity at every source:
- POST /agent/query no longer sets the prompt, the response or
  enduser.id. thread.id is recorded only for a caller-supplied thread
  id, because the default advisor_session_<user_id> embeds the user id.
- genai_span/record_completion, the inference proxy chat span, the NeMo
  rails and vLLM client, the advisor graph nodes and tool endpoint, MCP
  tracing and the MCP client record character lengths
  (cage.content.input_length, cage.content.output_length), never text
  or a hash of it.
- Governor spans record governance.tool and violation codes instead of
  the tool parameters; the post-HITL span drops the thread id.
- The NeMo block reason names the matched keyword instead of echoing
  the action result.

Add ContentFilterSpanExporter in
src/gateway/observability/content_filter.py and wrap every exporter that
configure_telemetry(), setup_tracing() and the evaluator auditor build.
It removes the denied exact keys, dotted prefixes, namespace-agnostic
suffixes and identity-bearing captured headers from span, event and
link attributes, and drops any span it cannot sanitise, logging only
counts. telemetry_client now forces GenAI content capture, httpx body
capture and TRACELOOP_TRACE_CONTENT off instead of forcing GenAI
content capture on.

Both workload modules set the three capture variables to "false" and no
longer capture the IAP end-user email header; a static test fails if
either module sets a capture variable to anything else.

Canary tests assert that no prompt, response or user id appears in raw
or exported span attributes, event attributes or resource attributes
for the advisor request span, the inference proxy span, genai_span, the
vLLM client and a LangGraph run under the LangChain instrumentation.

BREAKING CHANGE: telemetry spans no longer carry prompt or response content or end-user identifiers (gen_ai.input.messages, gen_ai.output.messages, Langfuse observation input/output, enduser.id); httpx body and GenAI content capture are off in both workload modules.
@lahlfors
lahlfors force-pushed the fix/stop-raw-prompt-export branch from f9c393d to 796a43c Compare November 16, 2026 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant