Add pod-based crank scheduler for simplified benchmark scheduling

README.md

@@ -15,3 +15,10 @@ The scenarios that are available to use with Crank and some sample command lines
 # Contributing
 
 Microsoft employees can approve and merge contributions by joining [this team](https://repos.opensource.microsoft.com/orgs/aspnet/teams/aspnet-push)
+
+# Continuous benchmarking pipelines
+
+The Azure DevOps pipelines that drive the daily benchmark sweep are generated
+from JSON configs by the [pod-scheduler](scripts/pod-scheduler/README.md).
+See [`build/README.md`](build/README.md) for the mapping between configs and
+generated YAML files, and how to regenerate them after editing.

build/README.md

@@ -0,0 +1,71 @@
+# `build/` — Pipeline definitions
+
+This directory contains Azure DevOps pipeline YAML files and the pod-scheduler
+configs that generate them.
+
+## Generated files (do not edit by hand)
+
+The CI pipelines that run the daily benchmark sweep are generated by the
+[pod-scheduler](../scripts/pod-scheduler/README.md). Each pipeline maps to a
+JSON config that lists the available pods (machine groups) and scenarios.
+
+| Generated YAML | Source config | Notes |
+|---|---|---|
+| `benchmarks-ci-01.yml`, `benchmarks-ci-02.yml` | `benchmarks_ci_pods.json` | On-prem fleet (citrine + mono); split across two YAMLs to spread load |
+| `benchmarks-ci-azure.yml` | `benchmarks_ci_azure_pods.json` | Azure cloud pods (includes the merged eastus2 cobalt cloud pods) |
+| `benchmarks-ci-cobalt.yml` | `benchmarks_ci_cobalt_pods.json` | Cobalt-hosted pods |
+
+To regenerate, from the repo root:
+
+```bash
+python ./scripts/pod-scheduler/main.py \
+    --config ./build/benchmarks_ci_pods.json \
+    --yaml-output ./build
+
+python ./scripts/pod-scheduler/main.py \
+    --config ./build/benchmarks_ci_azure_pods.json \
+    --base-name benchmarks-ci-azure \
+    --yaml-output ./build
+
+python ./scripts/pod-scheduler/main.py \
+    --config ./build/benchmarks_ci_cobalt_pods.json \
+    --base-name benchmarks-ci-cobalt \
+    --yaml-output ./build
+```
+
+The header of each generated YAML embeds the exact regen command for that
+file.
+
+## Hand-maintained scenario templates
+
+Everything else in this directory is hand-maintained. The most important
+groups:
+
+- `*-scenarios.yml` — AzDO job templates that the generated pipelines invoke
+  (e.g. `proxies-scenarios.yml`, `grpc-scenarios.yml`). One per benchmark
+  family. The pod-scheduler picks which template to run via the
+  `template` field on each scenario in the JSON config.
+- `prbenchmarks.*.config.yml` — PR-trigger pipelines for individual PR runs.
+- `job-variables.yml` — shared variable definitions consumed by the scenario
+  templates.
+
+When adding a new benchmark family, drop a `*-scenarios.yml` template in this
+directory and reference it from the relevant `benchmarks_ci_*_pods.json`.
+
+## Editing the schedule
+
+Common edits, all done in JSON:
+
+- **Add a scenario**: append to `scenarios[]` in the appropriate
+  `benchmarks_ci_*_pods.json`.
+- **Add a pod / machine class**: append to `pods[]` and reference its name
+  from any scenario you want to target it.
+- **Change cadence or splits**: edit `metadata.schedule` (cron),
+  `metadata.yaml_generation.target_yaml_count`, or
+  `metadata.yaml_generation.schedule_offset_hours`.
+- **Override a job's timeout**: add `"timeout": <minutes>` to the scenario.
+
+After any edit, regenerate the YAML and commit both the JSON and the YAML
+together. The snapshot tests in
+`scripts/pod-scheduler/tests/test_snapshots.py` will fail in CI if the
+generated YAML drifts from the configs.