eval: multi-grouping anomaly-detection templates (http, flows) for Hydra and roll-ups

[zzylol]

What

• Generator (scripts/export_autosketch_eval_table.py): a metric-level ordered label schema (SCHEMAS: labels with cardinality, child_of fan-out and Zipf skew; each value's data shape) and per-RQE grouping (label list) and covers_share (null = every group). One stream carries several groupings. The 10 templates are now the classic set; all = classic + templates 11, 12, 14, 15, 16, 17 on metrics http and flows, as the Hydra spec describes; template 17 (negative control: every group) groups by {service, endpoint}, whose smallest group holds 4.5e4 items in 5m. The tables carry schemas per workload. synthetic_plan() runs both sets over the grid (8 tables).
• Runner (autosketch_vs_asap.rs): for a schema stream, MetricFacts.labels = schema labels + x, cardinality for every used grouping and the full set (= series), data shape per grouping from the families' grid_param/grid_K; Raqe.grouping_labels from the RQE's grouping; capability cardinality maps to HLL. covers_share stays in the runner (Workload::covers_share), not on Raqe, and is reported in each choice, as is deployment_grouping when a deployment is finer than its RQE (a roll-up). Classic tables have neither field, so their output is unchanged.
• Coverage (review round 1, spec amendment): http labels are Zipf-skewed (region 0.5, service 1.1, endpoint 1.1 within its service, status 2.0) and the flows DDoS burst targets a tail subnet (999), so template 11 covers it. group_coverage() computes each group's share (product of its labels' shares; labels are independent given their parents) and the rule "share >= τ, plus always the largest group"; each schema RQE carries covered_groups, min_covered_share and covered_min_N; the generator fails if any covered_min_N is below the curves' first N (1e3). The runner reads a covered RQE's accuracy (ASAP and AutoSketch) at the smallest covered group's items per window, by scaling the series count in a per-RQE copy of its stream's facts (only items per group reads it); covers_share = null covers every group, so it is read at the smallest group of all (review round 2). Classic RQEs (no covers_share) keep the mean group's N. The per-RQE facts also go through validate_facts. The counted value's skew is unchanged.
• Worst covered group (sketch-bench#189: KLL/DD/HLL error does not fall as N grows, so the largest group can be the hardest): the generator also writes max_covered_share and covered_max_N (the largest group, always covered). The runner builds a second per-RQE facts copy at the largest covered group and holds ASAP, PerQuery and AutoSketch to the worse of the two reads in the family's direction (Workload::worse_read), unknown if either read is. Classic RQEs and Hydra's lookup are untouched; rqe-optimizer/src is unchanged.
• Plots: the synthetic script keys runs by (template set, shared, metrics); both scripts title classic "mixed" and all "mixed + multi-grouping".
• Results README: a "Template sets" section with the schema, rates, per-template groups, mean and smallest-covered items, which N the runner uses, and which schema fields are descriptive. label_set.keys_per_window is commented as the universe K.

Why

The 10 templates read each stream at one grouping, so roll-ups (#190) never trigger and Hydra has nothing to share. These anomaly-detection templates read one stream at many groupings.

Numbers: http region 4 x service 25 x endpoint 25/service x status 4 (1e4 combos, skewed as above), 2e6 samples/s like classic, user_id Zipf 0.8 over 1e6, latency Pareto a = 2. flows dst_subnet 1e3 (Zipf 1.1) x dst_port 1e3 (1.2) x proto 3, src_ip uniform over 1e6 plus a 5% DDoS burst from 1e4 sources at subnet 999; 4e6 samples/s (twice classic) so {dst_subnet, dst_port} (1e6 groups) holds 1,200 items per group in 5m, above the HLL curves' first N (1e3). Windows 5m and 15m, T = 1m: 54 new RQEs, 104 in all.

Tests

• python3 scripts/test_export_autosketch_eval_table.py (25 pass; new: a template whose smallest group is below 1e3 items (17 at the full label set) fails generation; label shares sum to 1 and the burst lifts subnet 999 to >= 5%; coverage equals a brute force over {region, status} at τ = 0.01/0.05/0.2, is the largest group alone at τ = 0.5, all groups at null; every schema RQE has >= 1 covered group and covered_min_N >= 1e3): classic tables equal the committed templatesall tables (main's) byte for byte after the rename; all = classic + 54; child labels count with their parents (endpoint 625); dropping labels never adds groups; every grouping has a series per group and >= 1e3 items in 5m; groupings are ordered, non-empty schema subsets (15 for template 15); entries carry grouping, coverage and shape; metric copies name data_i/http. Mutation-checked (no child closure; all without the new templates; grouping on classic entries; fixed shape): each fails a test.
• cargo test -p rqe-optimizer --example autosketch_vs_asap: new a_schema_stream_carries_every_grouping_and_rolls_up (two HLL RQEs, {region} and {region, service}: facts as above, and ASAP serves both from one {region, service} deployment). It now builds a temp saturation dir from the committed cost table and inline HLL curves, so it passes in a fresh checkout without the gitignored curves; it also checks the covered RQE's facts are scaled to 5% of the stream, and a null-coverage RQE's to its smallest group (each fails by mutation: scaling removed, null guard restored) and fails if the runner ignores grouping.
• Worst group: a_covered_rqe_is_held_to_the_worse_of_its_smallest_and_largest_group (RQE covering a 1% and a 50% group): with error rising in N, the smallest group meets a 3% target and the largest does not, and ASAP and AutoSketch read the largest; with error falling in N, the reverse. A classic RQE reads the mean group as before. Mutation: always smallest / always largest each fail it. Python: group_coverage returns the largest share, and every schema RQE has covered_max_N = rate·window·max_covered_share.
• cargo fmt --check, cargo clippy -p rqe-optimizer --all-targets -D warnings, cargo test -p rqe-optimizer, the example's tests and the Python tests: clean, in a fresh worktree without the curve CSVs.

Evidence

Inputs: #194's curves + the committed cost table; --runs 1.

• classic, r = 1: main's runner on main's table vs this branch's runner on the new templatesclassic table: outputs identical apart from timings and the workload name (50 RQEs, 0 dropped).

• all, r = 1, p95, freshly generated (--synthetic) with the committed saturation inputs (rqe-optimizer/results/autosketch-vs-asap-inputs/saturation; the committed templatesall tables are classic): 104 RQEs, 0 dropped, no sanity violations. CPU weights: ASAP 5.05 vCPU (15 active deployments), PerQuery 11.8, AutoSketch 3.42e3. ASAP serves 40 of the 54 new RQEs by roll-up: templates 14, 15 and 17 from HLL at {region, service, status} or the full label set, {dst_port} from {dst_port, proto}, p99 by {service} from DDSketch at {region, service} or {service, status}. AutoSketch and PerQuery: 0 roll-ups.

• Worst group (e7d8a03 vs 561ee86, all shared1, scratch saturation inputs without Hydra data): 104 RQEs, 0 dropped in both. Costs unchanged: CPU weights ASAP 5.779 (15 deployments), PerQuery 13.14, AutoSketch 3575; Fargate 0.6114 / 1.247 / 169. Same chosen configs for all three methods; only two DDSketch p99 RQEs' read rises (0.009645 to 0.009650, target 0.05). classic shared1: identical apart from timings.

The committed results were not rerun; their templatesall files are today's classic, so the plot scripts would now title them "mixed + multi-grouping" until they are regenerated under templatesclassic.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FhcWJExZcmVqS6r6rEtjis