You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 0965b48
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+10-9Lines changed: 10 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -79,17 +79,18 @@ cargo test --features otap-engine,stream-benchmark
79
79
## Benchmarks
80
80
81
81
The benchmark is the evidence that a function is cheap enough to leave on.
82
-
Expect three tiers:
82
+
Three tiers run in CI:
83
83
84
-
| Tier | When | Scope | Gate |State|
84
+
| Tier | When | Scope | Gate |Workflow|
85
85
| --- | --- | --- | --- | --- |
86
-
| Smoke | Every merge to `main`| Short run of each scenario with correctness checks | Fails on wrong answers | Planned |
87
-
| Nightly sweep | Every day | Full rate sweep per scenario | Alerts on a throughput or CPU regression beyond a set margin | The sweep runs today in `.github/workflows/asap-sketch-benchmark.yml` (nightly, on PRs that touch the code, and on manual dispatch); the regression alert is planned |
88
-
| PR comparison | On demand, by a comment on a PR | Same sweep, this PR vs `main`| Posts a comparison table | Planned |
89
-
90
-
Until the PR comparison exists, run the relevant benchmark yourself for a
91
-
performance-related change and paste the numbers into the PR. See
92
-
`benchmarks/` and the docs in `docs/`.
86
+
| Smoke | Every PR and every merge to `main`| fmt, clippy, tests, then `deploy/telemetry-functions/acceptance.py` on real collector containers | Fails on wrong answers |`ci.yml`|
87
+
| Nightly sweep | Every day | Full rate sweep per scenario | Fails when throughput drops more than the margin (default 10%) against the previous night |`asap-sketch-benchmark.yml`|
88
+
| PR comparison | A maintainer comments `/benchmark` on a PR | Reduced sweep, this PR vs `main`| Posts a comparison table |`benchmark-on-demand.yml`|
89
+
90
+
Capacity numbers need dedicated CPUs: the nightly and on-demand tiers run on
91
+
the runner named by the `BENCH_RUNNER` repository variable (see
92
+
`benchmarks/README.md`). For a performance-related change, also paste the
0 commit comments