tests: add MCP auth mode e2e collection and Newman runner for headers/both/oauth

[Pratham-Mishra04]

Summary

Adds an end-to-end Newman test suite that validates inbound /mcp authentication across all three server auth modes (headers, both, oauth). This ensures that enabling OAuth support does not regress existing virtual-key authentication behavior, and that each mode enforces exactly the access guarantees it is supposed to.

Changes

• Added bifrost-v1-mcp-auth.postman_collection.json — a Postman collection with two test groups:

  • Discovery gating: verifies that /.well-known/oauth-protected-resource, /.well-known/oauth-authorization-server, and /.well-known/jwks.json return 404 in headers mode and well-formed spec-compliant responses in both/oauth modes.
  • MCP connect matrix: exercises six credential scenarios (x-bf-vk, Authorization: Bearer <vk>, x-api-key, anonymous, inactive virtual key, and an invalid JWT) against each mode. A single collection encodes the full accept/reject matrix by branching on the auth_mode environment variable.

• Added run-newman-mcp-auth-tests.sh — a shell runner that:

  • Builds and starts the upstream http-no-ping-server MCP example so /mcp exposes real tools.
  • Boots a fresh Bifrost server per auth mode with a pre-seeded SQLite config containing one active and one inactive virtual key.
  • Runs the collection against each mode in sequence (headers → both → oauth) and aggregates pass/fail.
  • Supports --html, --json, --verbose, --bail, and --port flags for flexible CI integration.

Core guarantees validated:

1. In headers mode, all virtual-key credential paths connect exactly as before and the OAuth discovery surface is invisible (404).
2. In both mode, header-credential outcomes are unchanged and JWT acceptance is added.
3. In oauth mode, header credentials are rejected outright with a WWW-Authenticate response.
4. An inactive virtual key is rejected in every mode.
5. Discovery documents are spec-shaped when served.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Build the binary first, then run the test runner:

# Build the binary
go build -o bifrost-http ./cmd/bifrost-http

# Run the MCP auth e2e suite
cd tests/e2e/api
bash runners/individual/run-newman-mcp-auth-tests.sh \
  --binary ../../../bifrost-http \
  --port 8090

# Optional: generate HTML and JSON reports
bash runners/individual/run-newman-mcp-auth-tests.sh \
  --binary ../../../bifrost-http \
  --html --json

Newman must be installed (npm install -g newman). The runner will build the upstream MCP server from examples/mcps/http-no-ping-server automatically.

Expected outcome: all three modes pass with ✓ All MCP auth modes passed!.

Breaking changes

• No

Related issues

Security considerations

The test suite exercises the authentication boundary for the /mcp endpoint, including rejection of inactive credentials, invalid JWTs, and header-based credentials in strict OAuth mode. No secrets or PII are introduced; all keys used are test fixtures scoped to the local test server.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Pratham-Mishra04]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• tests: add MCP auth e2e collection covering discovery, credential matrix, OAuth issuance, refresh rotation, revocation window, and runtime mode flip #4525
• tests: add MCP auth mode e2e collection and Newman runner for headers/both/oauth #4524 👈 (View in Graphite)
• tests: add OAuth2 issuance, JWT verification, consent, discovery, sessions, and MCP auth unit tests #4523
• feat: add mcp_server_auth_mode and OAuth 2.1 gateway auth docs with gateway-auth page #4522
• ui: add OAuth Grants page and identity exact-match filter for MCP sessions #4511
• ui: add OAuth2 consent page and MCP server auth mode settings #4510
• feat: add OAuth2 session listing, revocation, family-revocation, VK liveness check, and sweep worker #4509
• feat: add JWT Bearer authentication path for /mcp with session validation and context injection #4508
• feat: add OAuth2ConsentHandler with VK, session, and user identity resolution modes #4507
• feat: add OAuth2 issuance endpoints (DCR, authorize, token) with PKCE and refresh token rotation #4506
• feat: add OAuth2 AS discovery endpoints, signing key management, and MCPServerAuthMode config #4505
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 06ee90a5-9a2a-4e79-9c68-6d26d9d94fe8

📥 Commits

Reviewing files that changed from the base of the PR and between ce7bb1b and 0fc852e.

📒 Files selected for processing (3)

• examples/mcps/http-no-ping-server/main.go
• tests/e2e/api/collections/bifrost-v1-mcp-auth.postman_collection.json
• tests/e2e/api/runners/individual/run-newman-mcp-auth-tests.sh

---

📝 Walkthrough

Summary by CodeRabbit

Release Notes

• Chores

  • The example MCP server now supports configurable listening port via the MCP_SERVER_PORT environment variable (defaults to 3001).

• Tests

  • Added comprehensive authentication test coverage for MCP endpoints across multiple authentication modes and credential types, including discovery endpoints and virtual key validation.

Walkthrough

Adds an e2e test suite for Bifrost MCP authentication. The upstream example HTTP MCP server gains a MCP_SERVER_PORT env-var override. A new Postman collection tests discovery endpoint gating and a credential matrix across three auth modes (headers, both, oauth). A Newman runner script provisions each mode with generated config, boots bifrost-http, runs the collection, and aggregates results.

Changes

MCP Auth E2E Test Suite

Layer / File(s)	Summary
Upstream MCP server configurable port examples/mcps/http-no-ping-server/main.go	Adds os import and replaces the fixed port 3001 with a MCP_SERVER_PORT environment variable override, enabling the test runner to assign the upstream port dynamically.
Postman collection: discovery gating and auth matrix tests/e2e/api/collections/bifrost-v1-mcp-auth.postman_collection.json	Defines the Bifrost V1 MCP Auth collection with shared variables, discovery endpoint gating tests (404 in headers mode, 200 with shape assertions in other modes), and a credential matrix of JSON-RPC initialize requests covering virtual key variants, anonymous, inactive key, and invalid JWT cases with mode-dependent status and WWW-Authenticate assertions.
Newman runner: CLI, cleanup trap, and upstream MCP startup tests/e2e/api/runners/individual/run-newman-mcp-auth-tests.sh (lines 1–156)	Parses CLI args, validates prerequisites (Newman, bifrost binary, collection file), registers an EXIT trap for process/temp-dir cleanup, and implements start_upstream_mcp() that builds the upstream server, starts it with MCP_SERVER_PORT set, and polls for readiness.
Newman runner: config generation, mode execution, and reporting tests/e2e/api/runners/individual/run-newman-mcp-auth-tests.sh (lines 158–288)	Implements write_config() to emit a per-mode config.json with auth-mode, optional oauth2 issuer settings, sqlite config-store, and seeded virtual keys. Implements run_mode() to boot bifrost-http, poll for readiness, run Newman with env vars and optional reporters, and accumulate exit codes. Main loop runs all three modes and exits with the aggregated status.

Sequence Diagram

sequenceDiagram
    participant Runner as run-newman-mcp-auth-tests.sh
    participant Upstream as http-no-ping-server
    participant Bifrost as bifrost-http
    participant Newman as newman

    Runner->>Upstream: build & start (MCP_SERVER_PORT override)
    Runner->>Runner: poll upstream port for readiness

    loop auth_mode ∈ {headers, both, oauth}
        Runner->>Runner: write_config(auth_mode) → config.json
        Runner->>Bifrost: start with config.json
        Runner->>Runner: poll logs for "successfully started bifrost"
        Runner->>Newman: run bifrost-v1-mcp-auth.postman_collection
        Newman->>Bifrost: GET /.well-known/* (discovery gating)
        Newman->>Bifrost: POST /mcp initialize (credential matrix)
        Bifrost-->>Newman: 200 / 401 + WWW-Authenticate
        Newman-->>Runner: exit code → OVERALL_EXIT
        Runner->>Bifrost: stop
    end

    Runner->>Runner: print summary, exit OVERALL_EXIT

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

• akshaydeo
• danpiths

Poem

🐇 Hop hop, the tests now run with flair,
Three auth modes checked with greatest care,
A Newman script, a collection new,
Discovery gates and credentials too.
The rabbit stamps the CI green —
The cleanest MCP auth ever seen! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and accurately summarizes the main changes: adding MCP auth mode e2e testing with a Postman collection and Newman runner covering three auth modes.
Description check	✅ Passed	The description is comprehensive and well-structured, covering summary, detailed changes, type of change, affected areas, testing instructions, security considerations, and checklist items with appropriate selections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 06-18-feat_adds_mcp_oauth_server_type_e2e_tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[greptile-apps]

Confidence Score: 5/5

Test-only additions with no changes to production code paths; safe to merge.

All three changed files are test infrastructure (a new Postman collection, a new Newman runner, and a one-line env-var addition to an example server). No production logic is touched. The collection and runner are well-structured, with proper process cleanup, port-conflict detection, and per-mode config isolation.

No files require special attention — all changes are confined to test tooling and an example server.

Important Files Changed

Filename	Overview
examples/mcps/http-no-ping-server/main.go	Adds MCP_SERVER_PORT env-var support so the test harness can bind to a non-default port; clean, minimal, correct change.
tests/e2e/api/collections/bifrost-v1-mcp-auth.postman_collection.json	New Postman collection covering discovery gating and the MCP connect accept/reject matrix across all three auth modes; notably omits a positive valid-JWT acceptance test for both/oauth modes.
tests/e2e/api/runners/individual/run-newman-mcp-auth-tests.sh	New shell runner that orchestrates upstream MCP server build, per-mode Bifrost boot with pre-seeded config, and Newman execution; --bail flag semantics diverge from its help text description.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Runner as run-newman-mcp-auth-tests.sh
    participant MCP as http-no-ping-server (upstream MCP)
    participant Bifrost as Bifrost HTTP Server
    participant Newman as Newman

    Runner->>MCP: go build → start (MCP_SERVER_PORT)
    MCP-->>Runner: ready on :3001

    loop For each mode: headers → both → oauth
        Runner->>Bifrost: boot with per-mode config.json
        Bifrost-->>Runner: successfully started

        Newman->>Bifrost: GET /.well-known/oauth-protected-resource
        Bifrost-->>Newman: "404 (headers) | 200 (both/oauth)"

        Newman->>Bifrost: POST /mcp x-bf-vk active-key
        Bifrost-->>Newman: "200 (headers/both) | 401 (oauth)"

        Newman->>Bifrost: POST /mcp x-bf-vk inactive-key
        Bifrost-->>Newman: 401 (all modes)

        Newman->>Bifrost: POST /mcp Bearer fake-jwt
        Bifrost-->>Newman: "200 anon (headers) | 401 (both/oauth)"

        Runner->>Bifrost: kill server
    end

    Runner-->>Runner: aggregate OVERALL_EXIT

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Runner as run-newman-mcp-auth-tests.sh
    participant MCP as http-no-ping-server (upstream MCP)
    participant Bifrost as Bifrost HTTP Server
    participant Newman as Newman

    Runner->>MCP: go build → start (MCP_SERVER_PORT)
    MCP-->>Runner: ready on :3001

    loop For each mode: headers → both → oauth
        Runner->>Bifrost: boot with per-mode config.json
        Bifrost-->>Runner: successfully started

        Newman->>Bifrost: GET /.well-known/oauth-protected-resource
        Bifrost-->>Newman: "404 (headers) | 200 (both/oauth)"

        Newman->>Bifrost: POST /mcp x-bf-vk active-key
        Bifrost-->>Newman: "200 (headers/both) | 401 (oauth)"

        Newman->>Bifrost: POST /mcp x-bf-vk inactive-key
        Bifrost-->>Newman: 401 (all modes)

        Newman->>Bifrost: POST /mcp Bearer fake-jwt
        Bifrost-->>Newman: "200 anon (headers) | 401 (both/oauth)"

        Runner->>Bifrost: kill server
    end

    Runner-->>Runner: aggregate OVERALL_EXIT

Loading

_{Reviews (3): Last reviewed commit: "feat: adds mcp oauth server type e2e tes..." | Re-trigger Greptile}