Repository navigation
Commit dae148f
Execute Planner physical operators through typed deployment bindings (#774)
* docs: clarify Planner physical plan and SDS architecture
* docs: specify executable subplan materialization boundaries
* docs: scope migration to backend precompute and query plans
* docs: clarify window terminology migration
* Revert "docs: clarify window terminology migration"
This reverts commit daa5281.
* docs: focus physical plan and SDS designs
* docs: add physical compiler input example
* docs: add physical compiler output example
* docs: include query expression in compiler example
* docs: reorganize physical plan integration design
* docs: reorganize SDS and migration designs
* docs: define maintenance inputs before plan split example
* docs: use plan version consistently in backend design
* docs: remove standalone catalog materialization abstraction
* docs: add concise planner backend glossary
* refactor: split installed maintenance DAGs from query execution
* refactor: remove backend Collector dependency and normalize legacy DAGs
* test: restore whole-backend process coverage without Collector
* fix: migrate Planner main and reject legacy runtime artifacts
* test: send full sketch envelope in whole-backend E2E
* refactor: bind backend summary state through versioned SDS slots
* test: send complete sketch envelopes in process fixtures
* docs: clarify selected deployment guarantee terminology
* docs: explain missing planner maintenance guarantee
* docs: motivate selected producer maintenance decision
* docs: label catalog reads and SDS metadata ownership
* docs: align SDS ownership and lifecycle terminology
* docs: use current planner and plan-version names consistently
* docs: align integration diagram with SDS ownership
* docs: clarify instance identity and shared producer wording
* refactor: align plan bindings with current Planner and SDS contract
* docs: distinguish summary definitions from runtime stores
* docs: model one runtime summary store for DAG bindings
* docs: scope SDS lifecycle to read eligibility
* docs: tie stored summary examples directly to DAG outputs
* docs: name summary tables, stored records, and output references by role
* docs: illustrate summary definitions, stored records, and output references
* docs: limit v1 summary storage to definitions and stored summaries
* refactor: align plan bindings with summary store v1
* fix: validate selected DAG provenance
* fix: retain neutral sketch codec dependencies when syncing main
* fix: align derived DAG validation with current schema versions
* fix: keep maintenance document version distinct from complete DAG version
* refactor: adopt costed Planner selection without legacy API adapters
* test: verify exact process routing for uncertified Planner candidates
* docs: remove redundant Planner selection contract
* deps: pin Planner bounded HLL confidence model
* test: size transmitted KLL state from a certified accuracy contract
* test: retain KLL collector capability when using theoretical confidence
* refactor: implement typed summary semantics and physical plan lowering
* docs: separate Planner physical computation from backend deployment
* refactor: name the backend orchestration entry DeploymentPlanCompiler
* Restack PR #770 with implementation before standalone acceptance
* chore: consume Planner physical precompute candidate interfaces
* chore: consume Planner materialization frontier enumeration
* chore: consume exact temporal ranking physical candidates
* chore: use shared Planner candidate winner selection
* chore: consume sparse counter shared readout contracts
* test: declare collector ranking fixture capabilities explicitly
* Use Planner counter-window candidate execution
* Use shared keyed-counter omission contract
* Use Planner exact-counter population omission
* Construct fixture evidence for the standalone operator foundation
* build: pin Planner exact-state scratch merge implementation
* build: pin shared finalized-pane reconstruction fix
* docs: bind Planner physical DAGs without backend re-lowering
* docs: describe summary inputs with groups and pane duration
* docs: define summary semantic completeness beyond input scope
* docs: define SDS identity through canonical Planner computation
* docs: decouple SDS semantic identity from executable Planner IR
* docs: define Planner-owned SDS discovery for future ad hoc queries
* docs: streamline SDS design around definitions and stored results
* docs: track bound-query SDS migration across implementation PRs
* docs: identify active shared-library PR in bound-query migration
* build: align shared Planner dependencies with remote PR 462
* docs: separate bound SDS range lookup from state validation
* feat(sds): separate bound output routing from persisted semantic identity
* docs: state SDS migration responsibilities without stale implementation claims
* refactor(sds): keep semantic variants compact without changing wire format
* test: align evidence fixtures with selected exact count state
* docs: align precompute SDS description with semantic catalog
* test: bind imported-state fixtures to their actual semantic definition
* test: distinguish imported CMS transport from total-count planning
* docs: clarify Planner physical plan and SDS architecture
* docs: specify executable subplan materialization boundaries
* docs: scope migration to backend precompute and query plans
* docs: clarify window terminology migration
* Revert "docs: clarify window terminology migration"
This reverts commit daa5281.
* docs: focus physical plan and SDS designs
* docs: add physical compiler input example
* docs: add physical compiler output example
* docs: include query expression in compiler example
* docs: reorganize physical plan integration design
* docs: reorganize SDS and migration designs
* docs: define maintenance inputs before plan split example
* docs: use plan version consistently in backend design
* docs: remove standalone catalog materialization abstraction
* docs: add concise planner backend glossary
* docs: clarify selected deployment guarantee terminology
* docs: explain missing planner maintenance guarantee
* docs: motivate selected producer maintenance decision
* docs: label catalog reads and SDS metadata ownership
* docs: align SDS ownership and lifecycle terminology
* docs: use current planner and plan-version names consistently
* docs: align integration diagram with SDS ownership
* docs: clarify instance identity and shared producer wording
* docs: distinguish summary definitions from runtime stores
* docs: model one runtime summary store for DAG bindings
* docs: scope SDS lifecycle to read eligibility
* docs: tie stored summary examples directly to DAG outputs
* docs: name summary tables, stored records, and output references by role
* docs: illustrate summary definitions, stored records, and output references
* docs: limit v1 summary storage to definitions and stored summaries
* docs: separate Planner physical computation from backend deployment
* docs: bind Planner physical DAGs without backend re-lowering
* docs: describe summary inputs with groups and pane duration
* docs: define summary semantic completeness beyond input scope
* docs: define SDS identity through canonical Planner computation
* docs: decouple SDS semantic identity from executable Planner IR
* docs: define Planner-owned SDS discovery for future ad hoc queries
* docs: streamline SDS design around definitions and stored results
* docs: track bound-query SDS migration across implementation PRs
* docs: identify active shared-library PR in bound-query migration
* docs: separate bound SDS range lookup from state validation
* docs: state SDS migration responsibilities without stale implementation claims
* docs: preserve design index after rebasing onto main
* docs: clarify SDS source identity and version-scoped recovery
* docs: illustrate SDS identity and recovery decisions
* feat: bind Planner dataset semantics to deployment input identity
* fix: recognize shared native batch encoding at dependency boundary
* test: reject pre-dataset planning snapshot versions
* refactor: version dataset-bound catalog and update empty-plan fixtures
* docs: describe dataset-bound planning and installation inputs
* docs: clarify candidate selection and deployment ownership
* refactor: split backend deployment plans and bind stored outputs
* fix: restrict foundation SDS recovery to the installed generation
* fix: allocate fresh physical series when a plan version changes
* test: record foundation rebase and recovery regression evidence
* fix: complete shared state encoding adoption at the dependency boundary
* style: satisfy workspace formatting after the compiler rename
* feat(sds): separate bound output routing from persisted semantic identity
* feat(sds): require dataset-bound Planner definitions and versioned catalogs
* refactor: implement typed summary semantics and physical plan lowering
* fix: validate final SDS semantics across Planner adapters and recovery
* test: verify final SDS identity installation, serving and restart
* test: align shared execution fixtures with final #749 SDS foundation
* docs: record shared execution directly after the SDS foundation
* fix(control-plane): supply dataset_identity in the API test fixtures
CompileAndPublishPhysicalPlanRequest gained a required `dataset_identity`
field in this branch, but two api_tests fixtures build their request body as
JSON by hand and were never updated, so both failed deserialization with
`missing field dataset_identity` before reaching the handler.
Take the value from the planning snapshot's own `environment.dataset_identity`
rather than inventing one, so the fixture keeps describing the same dataset the
rest of the snapshot describes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Remove obsolete SDS wire aliases and persisted metadata migration
* fix(data-plane): initialize dataset_identity in the bootstrap ingest contract
IngestContract gained an optional `dataset_identity`, but the bootstrap
envelope in data_plane's entry point was not updated, so the binary failed to
compile with E0063 and took the whole workspace build with it.
The bootstrap envelope describes the state before any plan is installed, where
every other field is a placeholder, so there is no dataset to bind to yet;
`None` is the accurate value. A published plan supplies the identity.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Reject obsolete state-column syntax in compiler roundtrip coverage
* Remove unused materialization wire adapter and propagate recovery errors
* Keep projection fixtures on the canonical typed encoding
* Remove superseded deployment API field aliases
* fix(control-plane): supply dataset_identity in the API test fixtures
CompileAndPublishPhysicalPlanRequest gained a required `dataset_identity`
field in this branch, but two api_tests fixtures build their request body as
JSON by hand and were never updated, so both failed deserialization with
`missing field dataset_identity` before reaching the handler.
Take the value from the planning snapshot's own `environment.dataset_identity`
rather than inventing one, so the fixture keeps describing the same dataset the
rest of the snapshot describes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit e627dbb)
* Document obsolete-format removals and cleanup validation
* Record passing restacked plan, storage and serving checks
* Remove generated evaluation archives and keep validation summaries
* Normalize validation summary formatting
* Remove PR process reports and defer execution design to shared-runtime PR
* Keep discovery and calibration on dataset-bound snapshot version 3
* Use typed deployment configuration in process E2E fixtures
* Align process assertions with dataset-bound SDS and cold successor activation
* fix: execute typed Planner fragments and preserve terminal resource errors
* fix: bind exact integer samples without losing input types
* test: verify exact integer protocol input binding
* refactor: name per-boundary physical fragments explicitly
* fix: consume finalized Planner query candidate outputs
* test: require Planner filters for bound protocol vectors
* test: use Planner schema lifting module
* fix: bind Planner filters and finalized query results
* docs: describe bound Planner filter execution
* fix: retain state binding and sharing beneath explicit query readouts
* fix: pin Planner query finalization for every candidate entry point
* docs: distinguish query values from stored accumulator boundaries
* test: assert exact Count state beneath its query readout
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent b08690b commit dae148f
97 files changed
Lines changed: 2201 additions & 13677 deletions
File tree
- control_plane
- examples
- src
- emit
- physical
- post_asap
- query_plan
- tests
- crates
- asap_sketch_codec
- src
- asap_types
- src
- query_plan
- data_plane
- benches
- examples
- src
- drivers
- ingest
- query/servers
- precompute_engine
- operators
- query_engines
- asap_clickhouse_query_engine
- asap_query_engine
- logical_dag
- routing
- storage_engines
- sketch_db
- backfill
- data
- index
- lifecycle
- persistence
- query
- types
- tests
- utils
- tests
- support
- docs
- design_docs
- developer_docs/control-plane
- scripts
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
7 | 6 | | |
8 | 7 | | |
9 | 8 | | |
| |||
15 | 14 | | |
16 | 15 | | |
17 | 16 | | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
22 | 21 | | |
23 | 22 | | |
24 | 23 | | |
| |||
38 | 37 | | |
39 | 38 | | |
40 | 39 | | |
| 40 | + | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
| |||
73 | 74 | | |
74 | 75 | | |
75 | 76 | | |
76 | | - | |
| 77 | + | |
77 | 78 | | |
78 | 79 | | |
79 | 80 | | |
80 | | - | |
| 81 | + | |
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| 59 | + | |
59 | 60 | | |
60 | 61 | | |
61 | 62 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2105 | 2105 | | |
2106 | 2106 | | |
2107 | 2107 | | |
| 2108 | + | |
2108 | 2109 | | |
2109 | 2110 | | |
2110 | 2111 | | |
| |||
2173 | 2174 | | |
2174 | 2175 | | |
2175 | 2176 | | |
2176 | | - | |
2177 | | - | |
2178 | | - | |
2179 | | - | |
2180 | | - | |
| 2177 | + | |
| 2178 | + | |
| 2179 | + | |
| 2180 | + | |
| 2181 | + | |
| 2182 | + | |
2181 | 2183 | | |
2182 | 2184 | | |
2183 | 2185 | | |
| |||
2380 | 2382 | | |
2381 | 2383 | | |
2382 | 2384 | | |
| 2385 | + | |
2383 | 2386 | | |
2384 | 2387 | | |
2385 | 2388 | | |
| |||
3612 | 3615 | | |
3613 | 3616 | | |
3614 | 3617 | | |
3615 | | - | |
3616 | 3618 | | |
| 3619 | + | |
3617 | 3620 | | |
3618 | 3621 | | |
3619 | 3622 | | |
3620 | 3623 | | |
3621 | 3624 | | |
3622 | | - | |
3623 | | - | |
3624 | | - | |
| 3625 | + | |
3625 | 3626 | | |
3626 | 3627 | | |
3627 | 3628 | | |
| |||
4490 | 4491 | | |
4491 | 4492 | | |
4492 | 4493 | | |
| 4494 | + | |
4493 | 4495 | | |
4494 | 4496 | | |
4495 | 4497 | | |
| |||
4517 | 4519 | | |
4518 | 4520 | | |
4519 | 4521 | | |
| 4522 | + | |
4520 | 4523 | | |
4521 | 4524 | | |
4522 | 4525 | | |
| |||
4541 | 4544 | | |
4542 | 4545 | | |
4543 | 4546 | | |
4544 | | - | |
| 4547 | + | |
4545 | 4548 | | |
4546 | 4549 | | |
4547 | 4550 | | |
| |||
5764 | 5767 | | |
5765 | 5768 | | |
5766 | 5769 | | |
| 5770 | + | |
| 5771 | + | |
| 5772 | + | |
| 5773 | + | |
| 5774 | + | |
5767 | 5775 | | |
5768 | 5776 | | |
5769 | 5777 | | |
| |||
5967 | 5975 | | |
5968 | 5976 | | |
5969 | 5977 | | |
5970 | | - | |
5971 | | - | |
5972 | | - | |
5973 | | - | |
5974 | | - | |
5975 | | - | |
5976 | | - | |
5977 | | - | |
5978 | | - | |
5979 | 5978 | | |
5980 | 5979 | | |
5981 | 5980 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
139 | 139 | | |
140 | 140 | | |
141 | 141 | | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
142 | 152 | | |
143 | 153 | | |
144 | 154 | | |
| |||
154 | 164 | | |
155 | 165 | | |
156 | 166 | | |
157 | | - | |
158 | 167 | | |
159 | 168 | | |
160 | 169 | | |
| |||
169 | 178 | | |
170 | 179 | | |
171 | 180 | | |
172 | | - | |
| 181 | + | |
173 | 182 | | |
174 | 183 | | |
175 | 184 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
177 | 189 | | |
178 | | - | |
179 | | - | |
| 190 | + | |
180 | 191 | | |
181 | 192 | | |
182 | 193 | | |
| |||
510 | 521 | | |
511 | 522 | | |
512 | 523 | | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
513 | 541 | | |
514 | 542 | | |
515 | 543 | | |
| |||
869 | 897 | | |
870 | 898 | | |
871 | 899 | | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
872 | 907 | | |
873 | 908 | | |
874 | 909 | | |
| |||
0 commit comments