Skip to content

fix(tests): close live tunnel and refuse GATEWAY_URL in hermetic runs - #766

Merged
lahlfors merged 1 commit into
mainfrom
fix/gate-w-tunnel-ordering
Nov 15, 2026
Merged

lahlfors merged 1 commit into
mainfrom
fix/gate-w-tunnel-ordering

Conversation

@lahlfors

Copy link
Copy Markdown
Collaborator

Summary

Stacked on #765 (fix/residency-lock -> #764 fix/region-resolver -> #762 -> #761 -> #760 -> #759).

scripts/staging_lifecycle.sh previously opened the staging kubectl port-forward tunnel during Phase 3 pre-Lula custody seeding (and again ahead of Phase 4) without closing it before Phase 4's us_fed, eu_ecb, and apac_mas hermetic posture suites, while tests/conftest.py defaulted GATEWAY_URL to http://localhost:8080 in every test session. The hermetic regional suites were therefore hermetic only because no selected test happened to read GATEWAY_URL; any hermetic test that did would hit live staging on localhost:8080 and land unmarked synthetic traffic in the deployed evidence chain and Lula counts.

Changes

  • Gate W Phase 3–5 tunnel ordering (scripts/staging_lifecycle.sh):
    • Stop Phase 3's port-forward when Lula validation finishes.
    • Run ensure_no_staging_port_forward (tearing down even a pre-existing external tunnel under --skip-lula) and pass env -u GATEWAY_URL in run_hermetic_region_pytest before executing the us_fed, eu_ecb, and apac_mas hermetic posture suites.
    • Open start_staging_port_forward only after the three hermetic regional suites complete—immediately before live data-residency (test_data_residency.py --run-integration), live Langfuse posture isolation (verify_langfuse_posture.py --posture staging), and Phase 5 live sweeps (also starting at Phase 5 entry when --skip-posture is passed).
  • Hermetic GATEWAY_URL guard (tests/fixtures/gateway_url_guard.py, tests/conftest.py):
    • Stop setting GATEWAY_URL="http://localhost:8080" at conftest.py import time and in hermetic pytest_configure sessions; populate the http://localhost:8080 default only when a live-run flag (--run-integration, --run-live-external, --run-partner-integration, --run-e2e) is active.
    • Refuse to start a hermetic pytest session (pytest.ExitCode.USAGE_ERROR) when GATEWAY_URL is exported in the environment.
    • Assert "GATEWAY_URL" not in os.environ before and after every hermetic test via the assert_hermetic_gateway_url_unset autouse fixture, failing closed and cleaning up if a test mutates os.environ["GATEWAY_URL"] without monkeypatch.setenv.
  • Tests & Documentation:
    • Add unit, regional-marker, and subprocess session/leak tests in tests/unit/test_gateway_url_guard.py.
    • Add stateful port-forward and GATEWAY_URL ordering/isolation tests across default, --skip-lula, and --skip-posture paths in tests/infrastructure/test_staging_lifecycle.py.
    • Update tests/README.md, docs/EVALUATOR_GUIDE.md, docs/operations/GKE_TEST_RUNBOOK.md, and CHANGELOG.md.

Verification

  • uv run pytest tests/unit/test_gateway_url_guard.py tests/test_dotenv_guard.py tests/test_deployment_region.py tests/infrastructure/test_staging_lifecycle.py -n 0 -q: 113 passed
  • uv run pytest tests/ -m "us_fed or eu_ecb or apac_mas" -n auto --dist loadscope --no-cov -q: 76 passed, 32 skipped
  • make test-fast (uv run pytest tests/ -m "local or unit" -n auto --dist loadscope --no-cov -p no:langsmith -q): 10,849 passed, 75 skipped, 6 subtests passed
  • uv run bandit -r src/ scripts/ -c pyproject.toml -ll -ii: 0 issues (135,986 lines scanned: src/ 107,976 lines, scripts/ 28,010 lines)
  • Deterministic repository gates (scripts/check_import_boundaries.py, scripts/check_domain_literals.py, scripts/check_vendor_brands.py, scripts/check_poam_lula_divergence.py, scripts/check_doc_references.py, scripts/check_project_id_hygiene.py): passed

scripts/staging_lifecycle.sh opened the staging kubectl port-forward
tunnel during Phase 3 pre-Lula custody seeding (and again before Phase
4) without closing it before Phase 4's us_fed, eu_ecb, and apac_mas
hermetic regional posture suites, while tests/conftest.py defaulted
GATEWAY_URL to http://localhost:8080 in every session. Any hermetic or
region-marked test reading GATEWAY_URL could therefore hit live staging
and record synthetic traffic in the deployed evidence chain and Lula
counts.

- Stop Phase 3's port-forward when Lula validation finishes, run
  ensure_no_staging_port_forward and pass env -u GATEWAY_URL before
  Phase 4's three hermetic regional suites, and start the port-forward
  only afterward (before live data-residency, live Langfuse isolation,
  and Phase 5 live sweeps).
- Add tests/fixtures/gateway_url_guard.py and wire it into
  tests/conftest.py so hermetic sessions stop defaulting GATEWAY_URL,
  refuse to start when GATEWAY_URL is exported, and assert
  "GATEWAY_URL" not in os.environ before and after every hermetic test;
  live sessions (--run-integration, --run-live-external,
  --run-partner-integration, --run-e2e) keep the localhost:8080 default.
- Add unit and subprocess contract tests in
  tests/unit/test_gateway_url_guard.py and port-forward / GATEWAY_URL
  isolation tests in tests/infrastructure/test_staging_lifecycle.py.
@lahlfors
lahlfors force-pushed the fix/gate-w-tunnel-ordering branch from cdb9c77 to 63af3d0 Compare November 15, 2026 18:53
@lahlfors
lahlfors changed the base branch from fix/residency-lock to main November 15, 2026 18:53
@lahlfors
lahlfors merged commit 64f811d into main Nov 15, 2026
42 checks passed
@lahlfors
lahlfors deleted the fix/gate-w-tunnel-ordering branch November 15, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant