[feature] support openrouter ori

docs/concepts.md

@@ -8,7 +8,7 @@ The mental model for benchflow. Read once, then refer back from the how-tos.
 | Primitive | What it is |
 |-----------|------------|
 | **Task** | A directory on disk: a `task.md` document (YAML frontmatter + prompt body) plus `environment/Dockerfile` for the sandbox, `verifier/` checks, and optional `oracle/` — or the legacy split layout (`task.toml` + `instruction.md` + `tests/` + `solution/`). Authored once, evaluated many times. |
-| **Agent** | A registered ACP-speaking program (Claude Code, Gemini CLI, OpenCode, etc.). Identified by name (`"gemini"`, `"opencode"`) plus an optional model ID. Use the `acpx/` prefix (e.g. `acpx/gemini`) to route through [ACPX](https://acpx.sh/), a headless ACP client with persistent sessions and crash recovery. |
+| **Agent** | A registered ACP-speaking program (Claude Code, Gemini CLI, OpenCode, Ori via BenchFlow's ACP shim, etc.). Identified by name (`"gemini"`, `"opencode"`, `"ori"`) plus an optional model ID. Use the `acpx/` prefix (e.g. `acpx/gemini`) to route through [ACPX](https://acpx.sh/), a headless ACP client with persistent sessions and crash recovery. |
 | **Environment** | The sandbox where the agent runs and the verifier checks the result. Docker locally, Daytona for cloud, Modal for serverless/GPU. Abstracted behind the `Sandbox` protocol — bring your own sandbox backend. |
 | **Verifier** | The test runner that scores the rollout. Its entry point is a `test.sh` script (native `verifier/test.sh`, legacy `tests/test.sh`) — which typically runs `pytest` against the workspace the agent left behind. For subjective tasks, use an [LLM-as-judge](./llm-judge.md) verifier scored against a rubric. Outputs `rewards: {reward: float}`. See the [verifier file map](#verifier-file-map) for which file lives where in native vs legacy packages. |
 | **Rollout** | One agent run on one task. Holds the lifecycle (setup → start → install → execute → verify → cleanup). All higher-level primitives below are built on Rollouts. |

docs/getting-started.md

@@ -49,19 +49,22 @@ uv sync --extra dev --locked
 
 ## Auth: OAuth, long-lived token, or API key
 
-You don't need an API key if you're a Claude / Codex / Gemini subscriber. Three options, pick one per agent:
+You don't need an API key if you're using a Claude / Codex / Gemini
+subscription or an OpenRouter login. Three options, pick one per agent:
 
 ### Option 1 — Subscription OAuth from host CLI login
 
 If you've logged into the agent's CLI on your host (`claude auth login`,
-`codex login`, or the `gemini` interactive flow), benchflow picks up the
-credential file and copies it into the sandbox. No API key billing.
+`codex login`, `ori login`, or the `gemini` interactive flow), benchflow picks up the
+credential file and copies it into the sandbox. No API key needs to be
+exported; billing and entitlements still follow the account used by that CLI.
 
 | Agent | How to log in on the host | What benchflow detects | Replaces env var |
 |-------|---------------------------|------------------------|------------------|
 | `claude-agent-acp` | `claude auth login` (Claude Code CLI) | `~/.claude/.credentials.json` | `ANTHROPIC_API_KEY` |
 | `codex-acp` | `codex login` (Codex CLI) | `~/.codex/auth.json` | `OPENAI_API_KEY` |
 | `gemini` | `gemini` (interactive login) | `~/.gemini/oauth_creds.json` | `GEMINI_API_KEY` |
+| `ori` | [`ori login`](https://openrouter.ai/docs/guides/ori/harness) | `~/.ori/credentials.json` or `~/.openrouter/credentials.json` | `OPENROUTER_API_KEY` |
 
 When benchflow finds the detect file, you'll see:
 
@@ -91,6 +94,7 @@ export ANTHROPIC_API_KEY=sk-ant-...
 export OPENAI_API_KEY=sk-...
 export CODEX_API_KEY=sk-...       # Codex alias for OPENAI_API_KEY
 export GEMINI_API_KEY=...
+export OPENROUTER_API_KEY=sk-or-... # OpenRouter / Ori
 export LLM_API_KEY=...           # OpenHands / LiteLLM-compatible providers
 export AZURE_API_KEY=...
 export AZURE_API_ENDPOINT='https://<resource>.openai.azure.com/'
@@ -163,6 +167,14 @@ GEMINI_API_KEY=... bench eval run \
 
 # List the registered agents
 bench agent list
+
+# Run OpenRouter's native Ori coding harness
+OPENROUTER_API_KEY=... bench eval run \
+  --source-repo benchflow-ai/skillsbench --source-path tasks \
+  --include citation-check \
+  --agent ori \
+  --model openrouter/anthropic/claude-sonnet-4.6 \
+  --sandbox docker
 ```
 
 `bench eval run` is the primary command for running evaluations — it works for

docs/reference/python-api.md

@@ -271,6 +271,7 @@ result = await bf.run(config)
 | `claude-agent-acp` | ACP | ANTHROPIC_API_KEY | `claude` |
 | `codex-acp` | ACP | OPENAI_API_KEY, CODEX_API_KEY, CODEX_ACCESS_TOKEN, or host login | `codex` |
 | `opencode` | ACP | inferred from model/provider | — |
+| `ori` | ACP (BenchFlow shim over Ori JSONL) | OPENROUTER_API_KEY or host login | — |
 | `openhands` | ACP | LLM_API_KEY | `oh` |
 | `pi-acp` | ACP | ANTHROPIC_API_KEY | `pi` |
 | `openclaw` | ACP | inferred from model | — |

docs/running-benchmarks.md

@@ -315,6 +315,7 @@ Common choices:
 | Gemini | `gemini` | `GEMINI_API_KEY` or host login |
 | Claude Code | `claude-agent-acp` (alias: `claude`) | `ANTHROPIC_API_KEY` or host login |
 | Codex | `codex-acp` (alias: `codex`) | `OPENAI_API_KEY`, `CODEX_API_KEY`, `CODEX_ACCESS_TOKEN`, or host login |
+| OpenRouter Ori | `ori` | `OPENROUTER_API_KEY` or `ori login` |
 | OpenHands | `openhands` (alias: `oh`) | `LLM_API_KEY` |
 | Harvey LAB harness | `harvey-lab-harness` (alias: `harvey-lab`) | Provider key matching model |
 
@@ -333,6 +334,35 @@ bench eval run --tasks-dir tasks/edit-pdf --agent acpx/gemini --model gemini-3.1
 ACPX is a headless ACP client that adds persistent sessions and crash recovery.
 The underlying agent's install, env vars, credentials, and skill paths are all preserved.
 
+### OpenRouter Ori harness
+
+`--agent ori` runs Ori's built-in coding harness through BenchFlow's ACP shim.
+The shim drives Ori's headless JSONL runtime, resumes Ori's native session id
+for follow-up turns, streams messages and tool calls as ACP events, and returns
+cumulative token usage in the ACP prompt response. BenchFlow pins and verifies
+the Ori binary before installing it.
+
+Use an OpenRouter API key:
+
+```bash
+export OPENROUTER_API_KEY=...
+bench eval run \
+  --tasks-dir tasks/citation-check \
+  --agent ori \
+  --model openrouter/anthropic/claude-sonnet-4.6 \
+  --sandbox docker
+```
+
+The first `openrouter/` selects BenchFlow's provider; the remainder is the
+OpenRouter model slug sent to Ori. For OpenRouter Auto, use
+`--model openrouter/openrouter/auto` (also Ori's registry default).
+
+Alternatively, install Ori on the host, run `ori login`, and unset
+`OPENROUTER_API_KEY`. BenchFlow detects either `~/.ori/credentials.json` or
+`~/.openrouter/credentials.json`, copies it into the sandbox, and uses Ori's
+native login. API-key runs go through BenchFlow's LiteLLM usage gateway; native
+login runs use the trusted token totals returned through the ACP shim.
+
 The **Harvey LAB harness** agent is special — it runs Harvey LAB's own agent loop
 (6 tools, system prompt) inside BenchFlow's sandbox. Use it for parity testing
 (same agent on both original and converted tasks).

src/benchflow/agents/env.py

@@ -388,8 +388,8 @@ def uses_native_subscription_auth(
     """Return True when an agent should use CLI/subscription auth directly.
 
     This is the Harbor-style split point: API-key runs can be routed through
-    LiteLLM, while subscription-auth runs stay on the native Codex/Claude ACP
-    path and report usage from the agent protocol response.
+    LiteLLM, while subscription-auth runs stay on the agent's native auth path
+    and report usage from the ACP response.
     """
     if agent_env.get("BENCHFLOW_PROVIDER_NAME") == "litellm" or any(
         agent_env.get(key) for key in _LITELLM_RUNTIME_MARKER_KEYS
@@ -414,28 +414,26 @@ def uses_native_subscription_auth(
             or check_subscription_auth(agent, required_key)
         )
 
-    # Registry-driven Claude-CLI gate: any agent whose subscription_auth
-    # substitutes ANTHROPIC_API_KEY runs the Claude Code CLI and can take
-    # OAuth/subscription auth natively (claude-agent-acp, omnigent claude-*).
-    claude_cfg = AGENTS.get(agent)
-    if (
-        claude_cfg is not None
-        and claude_cfg.subscription_auth is not None
-        and claude_cfg.subscription_auth.replaces_env == "ANTHROPIC_API_KEY"
-    ):
-        if agent_env.get("ANTHROPIC_API_KEY"):
+    # Registry-owned policy for native-login ACP agents. The containing
+    # SubscriptionAuth declares both the provider auth context (replaces_env)
+    # and any direct token aliases; no provider or agent name is hard-coded in
+    # this routing layer.
+    config = AGENTS.get(agent)
+    subscription = config.subscription_auth if config is not None else None
+    policy = subscription.native_policy if subscription is not None else None
+    if subscription is not None and policy is not None:
+        required_key = subscription.replaces_env
+        if agent_env.get(required_key):
             return False
         if model is not None:
             from benchflow.agents.registry import infer_env_key_for_model
 
-            if infer_env_key_for_model(model) != "ANTHROPIC_API_KEY":
+            if infer_env_key_for_model(model) != required_key:
                 return False
         return (
-            bool(agent_env.get(_CLAUDE_CODE_OAUTH_TOKEN_ENV))
-            or bool(agent_env.get(_CLAUDE_OAUTH_TOKEN_ENV))
-            or bool(agent_env.get("ANTHROPIC_AUTH_TOKEN"))
+            any(bool(agent_env.get(key)) for key in policy.direct_envs)
             or agent_env.get(_SUBSCRIPTION_AUTH_MARKER) == "1"
-            or check_subscription_auth(agent, "ANTHROPIC_API_KEY")
+            or check_subscription_auth(agent, required_key)
         )
 
     return False
@@ -558,7 +556,8 @@ def check_subscription_auth(agent: str, required_key: str) -> bool:
     sa = agent_cfg.subscription_auth
     if sa.replaces_env != required_key:
         return False
-    return Path(sa.detect_file).expanduser().is_file()
+    detect_files = sa.detect_files or [sa.detect_file]
+    return any(Path(path).expanduser().is_file() for path in detect_files)
 
 
 def validate_aws_bedrock_env(agent_env: dict[str, str], model: str) -> None: