Skip to content

eval: RQEs of one e2e query map to different deployments #214

Closed

Description

@milindsrivastava1997

Problem

Templates 14, 15 and 17 list some of the same groupings, so they produce identical queries with different accuracy scopes. For example, COUNT(DISTINCT user_id) … GROUP BY service, endpoint comes from t14 (groups ≥1%), t15 (≥5%) and t17 (every group).

#209 merges them into one e2e entry that reads all of their RQEs. The MILP still assigns each RQE separately, so one query can be mapped to two sketch deployments. However, ASAPQuery maps a query string to one aggregation (ProjectASAP/ASAPQuery#778), so --planner saved refuses these plans (ProjectASAP/ASAPQuery#856).

Affected: 8 multigroup queries (t14+t15 on region, service and region, service; t14+t15+t17 on service, endpoint; 5m and 15m each).

In #212's 10/s plans, 13 of the 16 asap SQL conversions of multigroup and all are refused. Example (multigroup, cpu, cheapest, 5m service, endpoint): t14 and t15 are on deployment 98 (HLL by region, service, endpoint), and t17 is on deployment 114 (HLL by service, endpoint, status). The 200/s plans happen not to split.

Options

  • The MILP requires the RQEs of one e2e query to share a deployment.
  • Give the duplicates distinct queries, or keep only one of them in the e2e workload.

Activity

  1. changed the title [-]eval: RQEs of one e2e query land on different deployments[/-] [+]eval: RQEs of one e2e query map to different deployments[/+] on Oct 14, 2026
  2. milindsrivastava1997 commented on Oct 14, 2026

    @milindsrivastava1997
    ContributorAuthor

    Decisions

    • Remove duplicates from the RQE list, then re-plan. Dropping them only from the e2e block would leave plan deployments the run never deploys, so measured cost wouldn't match the plan's prediction (docs(eval): plan the end-to-end evaluation (paper §6.2) ASAPQuery#862).
    • Keep the strictest copy of each query: covers_share none (every group), then 0.01, then 0.05. That's t17 for service, endpoint, t14 for region, service and region, service, and t15 elsewhere. The export errors if the strictest copy is left out at the table's rate; it doesn't fall back to a weaker copy.
    • On by default, with --keep-duplicate-queries to turn it off. The §6.3 commands in autosketch-vs-asap-hydra/README.md pass the flag, so those tables still reproduce byte for byte.
    • Order: a separate exporter PR on main. Once it and eval: instant RQEs read one scrape, not 1 s #215 have merged, the e2e 10/s tables are re-exported and re-planned once.
    • Done when every asap plan of the regenerated tables converts with ASAPQuery's --planner saved: SQL for multigroup and all (3 of 16 today), PromQL for classic and all. ASAPQuery#856 keeps refusing split queries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions