diff --git a/README.md b/README.md index 8567f45..1fb2841 100644 --- a/README.md +++ b/README.md @@ -167,6 +167,7 @@ Most "awesome" lists are link dumps. This one is **annotated and verified**: eve - **[Harbor](https://github.com/harbor-framework/harbor)** โ€” harbor-framework (Laude Institute / Stanford) โ€” โ€” ๐Ÿ†• framework for running agent evals + creating/using RL environments; powers Terminal-Bench 2.0. ~3.7kโ˜…. โš ๏ธ name overloaded (cf. `av/harbor` local-LLM toolkit). - **[Caliper](https://github.com/edonadei/caliper)** โ€” Emrick Donadei โ€” ยท โ€” ๐Ÿ†• **pass@k reliability harness for agent skills**: runs a skill k times against Claude Code / Codex / pi, grading each attempt with an LLM-autorater (`expect:`) and/or deterministic Python (`assert:`), then a `--baseline` re-run without the skill reports the delta to prove the skill beats the base agent. Git-diffable `.eval.yaml` specs; isolated per-attempt sandboxes. - **[Coder Eval](https://github.com/UiPath/coder_eval)** โ€” UiPath โ€” ยท โ€” ๐Ÿ†• treats **skill activation as a measurable variable**: `skill_triggered` over a labelled dataset yields suite-level precision/recall/F1 gates (cf. ยง3). Harness-level, not leaderboard-level. ~0.1kโ˜…. +- **[AgentRunProof](https://github.com/FU-max-boop/agentrunproof)** โ€” Fu Xiaonan โ€” ยท ยท *tool/repo* โ€” ๐Ÿ†• Provider-free OpenAI Agents runtime-regression harness: deterministic scripts drive the real `Runner`, compare `run` / `run_streamed` and `RunState` resume invariants, and emit content-addressed records; the case study follows the public evidence chain through merged upstream fixes [#4413](https://github.com/openai/openai-agents-python/pull/4413) and [#4414](https://github.com/openai/openai-agents-python/pull/4414). โš ๏ธ Tests SDK runtime correctness, not model quality; independent project, not OpenAI adoption or endorsement. ### 5b ยท TypeScript/JS-native eval runners - **[evalite](https://github.com/mattpocock/evalite)** โ€” Matt Pocock โ€” โ€” ๐Ÿ†• local-first eval runner on Vitest; `.eval.ts` files, web UI, cost-aware.