Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,9 +293,9 @@ section, when present, overrides the first two for backward compatibility.

The unified nginx endpoint is same-origin by default and does not emit browser CORS headers. If you run a split-origin or port-forwarded browser client, set `GATEWAY_CORS_ORIGINS` to comma-separated exact origins such as `http://localhost:3000`; the Gateway then applies the CORS allowlist and matching CSRF origin checks.

Browser login uses `HttpOnly` session cookies. The login page offers a "keep me signed in" option that extends the browser session when the request is HTTPS (including trusted `X-Forwarded-Proto: https`) or localhost HTTP. The localhost exception uses the direct request `Host` and ignores forwarded host headers. Public HTTP deployments, including many temporary sandbox URLs, fall back to session cookies by default. DeerFlow never stores the password in browser storage; the UI may remember only the email address.
Browser login uses `HttpOnly` session cookies. The login page offers a "keep me signed in" option that extends the browser session when the request is HTTPS (including `X-Forwarded-Proto: https` from a TCP peer allowed by `AUTH_TRUSTED_PROXIES`) or localhost HTTP. The localhost exception uses the direct request `Host` and ignores forwarded host headers. Public HTTP deployments, including many temporary sandbox URLs, fall back to session cookies by default. DeerFlow never stores the password in browser storage; the UI may remember only the email address.

DeerFlow still uses `Forwarded` / `X-Forwarded-*` headers to recover the browser-facing scheme and origin behind a proxy. The bundled nginx sets `X-Forwarded-Proto`, but preserves an upstream HTTPS value and does not overwrite every forwarded header. Configure the outer trusted proxy to replace or strip client-supplied forwarding headers before traffic reaches DeerFlow.
DeerFlow uses `Forwarded` / `X-Forwarded-*` headers to recover the browser-facing scheme and origin only when the TCP peer is trusted by `AUTH_TRUSTED_PROXIES`. The bundled nginx sets `X-Forwarded-Proto`, but preserves an upstream HTTPS value and does not overwrite every forwarded header. Configure the outer trusted proxy to replace or strip client-supplied forwarding headers before traffic reaches DeerFlow.

> [!IMPORTANT]
> The Gateway still owns active run tasks in process, so production defaults to a single Gateway worker (`GATEWAY_WORKERS=1`). Multi-worker deployments require Postgres, the Redis stream bridge (`stream_bridge.type: redis`), `run_ownership.heartbeat_enabled: true`, and `run_events.backend: db`; process-local memory/JSONL event stores cannot enforce singleton delivery receipts across workers. The bridge shares SSE delivery and bounded `Last-Event-ID` replay across workers. When a valid reconnect cursor has been trimmed, or a subscriber that already established an empty-stream wait falls behind before its first delivery, Memory and Redis emit a machine-readable SSE `gap` event instead of silently returning a partial replay; the Web UI reloads durable thread/event state and resumes from the retained tail. Lease reconciliation marks runs from dead workers as errors, persists their delivery receipts, publishes the terminal stream marker, schedules retained-stream cleanup, and updates the affected thread status. SSE and `/wait` consumers also refresh durable status on heartbeats as a fallback if terminal publication fails. Malformed Redis reconnect IDs live-tail new events instead of replaying the retained buffer, and the rolling retained-buffer TTL (`stream_ttl_seconds`) remains a cleanup safety net rather than a run timeout. IM channel state and other process-local services still need their own multi-worker coordination.
Expand Down
2 changes: 1 addition & 1 deletion backend/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -545,7 +545,7 @@ CORS is same-origin by default when requests enter through nginx on port 2026. S

Browser auth sessions are owned by `app.gateway.auth.session_cookie`. Login accepts a `remember_me` form flag, but the Gateway never stores passwords. `SessionCookiePolicy` persists the `HttpOnly access_token` cookie only for HTTPS/trusted-forwarded HTTPS, direct-host localhost HTTP, or explicit operator opt-in for insecure persistence; public HTTP sandbox URLs degrade to session cookies. Session-creating handlers stamp the final `max_age` on `request.state`, and CSRF cookie creation mirrors that value so the double-submit cookie pair expires together, including explicit re-issue after password changes and OIDC callbacks. A small `HttpOnly` preference cookie preserves the user's remember choice across token re-issue paths. Logout clears all auth cookies and suppresses CSRF re-issue on the logout response.

Localhost persistence deliberately reads the direct request `Host` and ignores `Forwarded` / `X-Forwarded-Host`. Scheme and auth-origin reconstruction still consume forwarding headers. The bundled nginx sets `X-Forwarded-Proto`, but preserves an upstream HTTPS value and does not overwrite every forwarded header, so the outer trusted proxy must replace or strip client-supplied forwarding headers before traffic reaches DeerFlow.
Localhost persistence deliberately reads the direct request `Host` and ignores `Forwarded` / `X-Forwarded-Host`. Scheme and auth-origin reconstruction consume `Forwarded` / `X-Forwarded-*` only when the TCP peer matches `AUTH_TRUSTED_PROXIES`; direct requests ignore spoofed forwarding headers. The bundled nginx sets `X-Forwarded-Proto`, but preserves an upstream HTTPS value and does not overwrite every forwarded header, so the outer trusted proxy must replace or strip client-supplied forwarding headers before traffic reaches DeerFlow.

**Routers**:

Expand Down
39 changes: 34 additions & 5 deletions backend/app/gateway/csrf_middleware.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import os
import secrets
from collections.abc import Awaitable, Callable
from ipaddress import ip_address, ip_network
from urllib.parse import urlsplit

from fastapi import Request, Response
Expand Down Expand Up @@ -150,20 +151,48 @@ def _forwarded_param(request: Request, name: str) -> str | None:
return None


def _trusted_proxy_networks() -> list:
nets = []
for entry in os.getenv("AUTH_TRUSTED_PROXIES", "").split(","):
entry = entry.strip()
if not entry:
continue
try:
nets.append(ip_network(entry, strict=False))
except ValueError:
continue
return nets


def _trust_forwarded_headers(request: Request) -> bool:
peer_host = getattr(getattr(request, "client", None), "host", None)
if not peer_host:
return False
try:
peer_ip = ip_address(peer_host)
except ValueError:
return False
return any(peer_ip in net for net in _trusted_proxy_networks())


def _request_scheme(request: Request) -> str:
"""Resolve the original request scheme from trusted proxy headers."""
scheme = _forwarded_param(request, "proto") or _first_header_value(request.headers.get("x-forwarded-proto")) or request.url.scheme
scheme = request.url.scheme
if _trust_forwarded_headers(request):
scheme = _forwarded_param(request, "proto") or _first_header_value(request.headers.get("x-forwarded-proto")) or scheme
return scheme.lower()


def _request_origin(request: Request) -> str | None:
"""Build the origin for the URL the browser is targeting."""
scheme = _request_scheme(request)
host = _forwarded_param(request, "host") or _first_header_value(request.headers.get("x-forwarded-host")) or request.headers.get("host") or request.url.netloc
host = request.headers.get("host") or request.url.netloc

forwarded_port = _first_header_value(request.headers.get("x-forwarded-port"))
if forwarded_port and ":" not in host.rsplit("]", 1)[-1]:
host = f"{host}:{forwarded_port}"
if _trust_forwarded_headers(request):
host = _forwarded_param(request, "host") or _first_header_value(request.headers.get("x-forwarded-host")) or host
forwarded_port = _first_header_value(request.headers.get("x-forwarded-port"))
if forwarded_port and ":" not in host.rsplit("]", 1)[-1]:
host = f"{host}:{forwarded_port}"

return _normalize_origin(f"{scheme}://{host}")

Expand Down
22 changes: 19 additions & 3 deletions backend/tests/test_auth_type_system.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,13 @@
# ── Setup ────────────────────────────────────────────────────────────

_TEST_SECRET = "test-secret-for-auth-type-system-tests-min32"
_TRUSTED_PROXY_HOST = "10.0.0.2"


@pytest.fixture(autouse=True)
def _trusted_proxy_env(monkeypatch):
"""Use a trusted proxy by default for forwarded-header auth tests."""
monkeypatch.setenv("AUTH_TRUSTED_PROXIES", "10.0.0.0/8")


@pytest.fixture(autouse=True)
Expand Down Expand Up @@ -514,9 +521,9 @@ def _make_auth_app():
return create_app()


def _get_auth_client():
def _get_auth_client(*, client_host: str = _TRUSTED_PROXY_HOST):
"""Get TestClient for auth API contract tests."""
return TestClient(_make_auth_app())
return TestClient(_make_auth_app(), client=(client_host, 12345))


def test_api_auth_me_no_cookie_returns_structured_401():
Expand Down Expand Up @@ -638,7 +645,13 @@ def _get_response_set_cookie_headers(resp) -> list[str]:
return [v.decode("latin-1") for k, v in resp.raw_headers if k.lower() == b"set-cookie"]


def _make_request_scope(*, scheme: str = "http", host: str = "example.test", headers: dict[str, str] | None = None) -> dict:
def _make_request_scope(
*,
scheme: str = "http",
host: str = "example.test",
headers: dict[str, str] | None = None,
client_host: str = _TRUSTED_PROXY_HOST,
) -> dict:
raw_headers = [(b"host", host.encode("ascii"))]
for key, value in (headers or {}).items():
raw_headers.append((key.lower().encode("ascii"), value.encode("ascii")))
Expand All @@ -649,6 +662,7 @@ def _make_request_scope(*, scheme: str = "http", host: str = "example.test", hea
"headers": raw_headers,
"scheme": scheme,
"server": (host.split(":", 1)[0], 80 if scheme == "http" else 443),
"client": (client_host, 12345),
"query_string": b"",
}

Expand Down Expand Up @@ -1113,6 +1127,7 @@ def test_oidc_callback_access_and_csrf_cookie_lifetime_match_on_https():
"headers": [(b"x-forwarded-proto", b"https")],
"scheme": "http",
"server": ("internal", 8000),
"client": (_TRUSTED_PROXY_HOST, 12345),
"query_string": b"",
}
response = Response()
Expand Down Expand Up @@ -1143,6 +1158,7 @@ def test_oidc_callback_access_and_csrf_cookie_stay_session_only():
"headers": [(b"x-forwarded-proto", b"https")],
"scheme": "http",
"server": ("internal", 8000),
"client": (_TRUSTED_PROXY_HOST, 12345),
"query_string": b"",
}
response = Response()
Expand Down
31 changes: 26 additions & 5 deletions backend/tests/test_csrf_middleware.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,9 @@ def test_auth_post_allows_same_origin_default_port_equivalence():
assert response.cookies.get("csrf_token")


def test_auth_post_allows_forwarded_same_origin():
client = TestClient(_make_app(), base_url="http://internal:8000")
def test_auth_post_allows_forwarded_same_origin(monkeypatch):
monkeypatch.setenv("AUTH_TRUSTED_PROXIES", "10.0.0.0/8")
client = TestClient(_make_app(), base_url="http://internal:8000", client=("10.0.0.2", 12345))

response = client.post(
"/api/v1/auth/login/local",
Expand All @@ -110,9 +111,28 @@ def test_auth_post_allows_forwarded_same_origin():
assert response.cookies.get("csrf_token")


def test_auth_post_allows_forwarded_same_origin_with_non_default_port():
def test_auth_post_rejects_spoofed_forwarded_same_origin_without_trusted_proxy(monkeypatch):
monkeypatch.delenv("AUTH_TRUSTED_PROXIES", raising=False)
client = TestClient(_make_app(), base_url="http://internal:8000")

response = client.post(
"/api/v1/auth/login/local",
headers={
"Origin": "https://deerflow.example",
"X-Forwarded-Proto": "https",
"X-Forwarded-Host": "deerflow.example",
},
)

assert response.status_code == 403
assert response.json()["detail"] == "Cross-site auth request denied."
assert response.cookies.get("csrf_token") is None


def test_auth_post_allows_forwarded_same_origin_with_non_default_port(monkeypatch):
monkeypatch.setenv("AUTH_TRUSTED_PROXIES", "10.0.0.0/8")
client = TestClient(_make_app(), base_url="http://internal:8000", client=("10.0.0.2", 12345))

response = client.post(
"/api/v1/auth/login/local",
headers={
Expand All @@ -126,8 +146,9 @@ def test_auth_post_allows_forwarded_same_origin_with_non_default_port():
assert response.cookies.get("csrf_token")


def test_auth_post_allows_rfc_forwarded_same_origin():
client = TestClient(_make_app(), base_url="http://internal:8000")
def test_auth_post_allows_rfc_forwarded_same_origin(monkeypatch):
monkeypatch.setenv("AUTH_TRUSTED_PROXIES", "10.0.0.0/8")
client = TestClient(_make_app(), base_url="http://internal:8000", client=("10.0.0.2", 12345))

response = client.post(
"/api/v1/auth/login/local",
Expand Down
30 changes: 25 additions & 5 deletions backend/tests/test_oidc_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -360,12 +360,12 @@ async def fake_get(url):


def _redirect_request(headers: dict, scheme: str = "http", netloc: str = "localhost:8001"):
from unittest.mock import MagicMock
from types import SimpleNamespace

req = MagicMock()
req = SimpleNamespace()
req.headers = headers
req.url.scheme = scheme
req.url.netloc = netloc
req.url = SimpleNamespace(scheme=scheme, netloc=netloc)
req.client = SimpleNamespace(host="testclient")
return req


Expand All @@ -378,9 +378,10 @@ def test_oidc_redirect_uri_prefers_configured_value():
assert _resolve_oidc_redirect_uri(req, "keycloak", cfg) == "https://app.example.com/api/v1/auth/callback/keycloak"


def test_oidc_redirect_uri_fallback_uses_forwarded_headers_not_raw_host():
def test_oidc_redirect_uri_fallback_uses_forwarded_headers_from_trusted_proxy(monkeypatch):
from app.gateway.routers.auth import _resolve_oidc_redirect_uri

monkeypatch.setenv("AUTH_TRUSTED_PROXIES", "10.0.0.0/8")
cfg = _provider_config()
# Raw Host is attacker-controlled; proxy-set X-Forwarded-* must win.
req = _redirect_request(
Expand All @@ -390,12 +391,31 @@ def test_oidc_redirect_uri_fallback_uses_forwarded_headers_not_raw_host():
"x-forwarded-proto": "https",
}
)
req.client.host = "10.0.0.2"

result = _resolve_oidc_redirect_uri(req, "keycloak", cfg)

assert result == "https://app.example.com/api/v1/auth/callback/keycloak"


def test_oidc_redirect_uri_fallback_ignores_forwarded_headers_from_untrusted_peer(monkeypatch):
from app.gateway.routers.auth import _resolve_oidc_redirect_uri

monkeypatch.delenv("AUTH_TRUSTED_PROXIES", raising=False)
cfg = _provider_config()
req = _redirect_request(
{
"host": "localhost:8001",
"x-forwarded-host": "attacker.example.com",
"x-forwarded-proto": "https",
}
)

result = _resolve_oidc_redirect_uri(req, "keycloak", cfg)

assert result == "http://localhost:8001/api/v1/auth/callback/keycloak"


def test_oidc_redirect_uri_fallback_plain_host_when_no_proxy_headers():
from app.gateway.routers.auth import _resolve_oidc_redirect_uri

Expand Down