Summary
28 vectors define agent tool-calling behaviour and no runtime has ever passed them. The emitter-adjacent defect they expose (see companion issue) was reachable the entire time; no backend test matrix was in a position to report it.
| runtime |
references agent_vectors |
status |
| rust |
present |
0 passed / 28 failed |
| go |
absent |
suite not ported |
| typescript |
absent |
suite not ported |
| python |
present |
never executed (package does not build) |
| csharp |
present |
never executed |
Companion: a measurement trap that hid it
Plain cargo test stops after the first failing target, so it only ever reported the lib target. Every "293 passed / 3 failed" figure quoted during the 0.4.19-0.4.21 effort was lib-only and wrong.
Measured on measure/wip-conn (pinned to published @typra/emitter 0.4.21):
cargo test -> 293 passed; 3 failed (lib target only)
cargo test --no-fail-fast -> 39 failures across 7 targets
Breakdown of the 39:
lib 3 failed
agent_vectors 28 failed
named_collection_vectors 3 failed (all: model: missing required field)
+ 4 more single-test targets 5 failed
Any Rust measurement in this repo needs --no-fail-fast. Worth encoding in CI and in the contributing docs.
Requests
- Port
agent_vectors to Go and TypeScript.
- Get Python and C# executing the vectors they already reference (Python is blocked on the package not building).
- Add
--no-fail-fast to the Rust CI invocation so target-level failures cannot be masked.
Summary
28 vectors define agent tool-calling behaviour and no runtime has ever passed them. The emitter-adjacent defect they expose (see companion issue) was reachable the entire time; no backend test matrix was in a position to report it.
agent_vectorsCompanion: a measurement trap that hid it
Plain
cargo teststops after the first failing target, so it only ever reported the lib target. Every "293 passed / 3 failed" figure quoted during the 0.4.19-0.4.21 effort was lib-only and wrong.Measured on
measure/wip-conn(pinned to published@typra/emitter0.4.21):Breakdown of the 39:
Any Rust measurement in this repo needs
--no-fail-fast. Worth encoding in CI and in the contributing docs.Requests
agent_vectorsto Go and TypeScript.--no-fail-fastto the Rust CI invocation so target-level failures cannot be masked.