diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c19ecde4..3d66c6f8b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,8 @@ ## [Unreleased] +## 0.7.6 — 2026-09-04 + ### Added - **ACP rollout directories render as an interactive reviewer page.** `bench eval view ` now assembles a JSON payload (normalized @@ -42,6 +44,48 @@ timestamps.** Steps gain a `+m:ss` offset chip and tool calls a duration chip whenever events carry `ts` / `started_at` / `finished_at` fields; with today's captures (which carry none) nothing changes visually. +- **Z.ai Coding Plan routing.** Coding Plan subscriptions route through the + provider layer, with validated generation parameters forwarded and explicit + provider endpoints preserved by the LiteLLM proxy. (#1074) +- **`bench eval run` publishes job artifacts and eval results.** + `--publish-bucket` syncs a job directory to a HuggingFace storage bucket (an + alternative to `--publish-hf`'s dataset-repo upload), and + `--eval-results-model/-dataset/-task` opens a community eval-results PR + (`.eval_results/*.yaml`) on a model's HF repo, scored from the run's mean + reward. (#1035) + +### Fixed + +- **`claude-fable-5-1` can run: the `claude-agent-acp` pin moves 0.40.0 to + 0.73.0.** The old pin bundled `@anthropic-ai/claude-agent-sdk` 0.3.160, and + the model rejects Claude Code older than 2.1.251 with + `claude_code_version_too_old` (HTTP 400) — so every rollout failed on its + first API call. The new pin bundles sdk 0.3.257, and the `set_config_option` + `"model"` / `"effort"` wiring is re-verified against it by + `tests/test_acp_pinned_protocol_guard.py`. (#1086) +- **`codex-acp` dispatch modernized.** The shim pin moves to 1.6.0 and + reasoning effort travels inside the model id (`modelId[effort]`) through + `session/set_model`, rather than a config option the shim rejects. (#1044) +- **ACP usage and terminal evidence survive an agent timeout**, instead of + being dropped with the timed-out turn. (#1080) +- **A pending tool call can no longer defer the idle watchdog without + bound** — the deferral is capped. (#1066) +- **`bench eval` resume re-runs infra-retryable verifier-errored tasks** + rather than treating the infrastructure failure as a settled result. (#1063) +- **Sandbox hardening execs draw on the verifier-setup budget**, so hardening + no longer competes with the agent's own time. (#1062) +- **`--trials > 1` without `--matrix` is rejected up front.** (#1064) +- **Healthy native ACP subscription results are preserved.** (#1049) +- **The agent judge parses native edit targets.** (#1069) +- **Non-dict rewards no longer break completed-outcome classification.** (#1054) +- **Codex honors proxy-owned model selection.** (#1076) +- **Gemini routing corrections.** ACP model ids fixed and wrapped ids + normalized, Gemma routing supported with pass-through usage captured, every + Google key alias routed through the proxy, Google Gemini gateway routes + normalized, and headless runs trust the sandbox workspace. +- **LiteLLM provisions the Vertex sandbox runtime.** (#985) +- **OpenClaw caps supported model output tokens.** +- **RestrictedPython updated to 8.3.** ## 0.7.5 — 2026-08-19 diff --git a/pyproject.toml b/pyproject.toml index 2c1753f80..4efed52c9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "benchflow" -version = "0.7.6.dev0" +version = "0.7.6" description = "Multi-turn agent benchmarking with ACP — run any agent, any model, any provider." readme = "README.md" requires-python = ">=3.12" diff --git a/uv.lock b/uv.lock index f9b03f128..6a8f46915 100644 --- a/uv.lock +++ b/uv.lock @@ -362,7 +362,7 @@ wheels = [ [[package]] name = "benchflow" -version = "0.7.6.dev0" +version = "0.7.6" source = { editable = "." } dependencies = [ { name = "agent-client-protocol" },