Repository navigation
fix(types): reject a query-time summary build over an unstored ingestion-time input - #746
Merged
Merged
Conversation
zzylol
force-pushed
the
fix/query-time-summary-agg-unstored-input
branch
from
October 18, 2026 15:37
888c42a to
c342271
Compare
…ion-time input A query-time SummaryAgg or PaneChain whose child is ingestion-time rows or exact-accumulator state that is not MatAtIngestion validated, although at query time that input no longer exists. Apply the NonASAP stored-output rule to the summary build's child edge and report the new ExecutionDataStateError::UnstoredIngestionInputUnderQueryTimeSummary. The edge label of the build's errors now names PaneChain.child for a PaneChain. Closes #738
zzylol
force-pushed
the
fix/query-time-summary-agg-unstored-input
branch
from
October 18, 2026 15:42
c342271 to
376a2b3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A query-time summary build over an ingestion-time input that nobody stores passed timing validation, although the plan cannot run. At query time that input no longer exists, and the query-time build cannot see the ingestion stream (#738).
The same holds when the input is ingestion-time exact-accumulator state, e.g. an exact-sum
SummaryAggpinned to ingestion time. Today that case is caught only later, at export, with an unrelatedMaterializationTimingMismatch.What
validate_asapnow rejects a query-timeSummaryAggorPaneChainwhose child produces its output at ingestion time (rows or exact-accumulator state) and is notMatAtIngestion. It reports the new errorExecutionDataStateError::UnstoredIngestionInputUnderQueryTimeSummary, which names the edge and the input node's operator.MatAtIngestionstill validates: the query-time build reads the stored output.QueryTimeValueUnderIngestionSummaryerrors now namePaneChain.childfor aPaneChain; before, they always saidSummaryAgg.child.timingmodule docs gains the new rule forSummaryAgg.child/PaneChain.child.How
The
SummaryAgg.child/PaneChain.childarm ofvalidate_asap(crates/types/src/ir/properties/timing.rs) gets the stored-output check theNonASAPconsumer rule already uses (reads_stored:child.materialization.timing() == IngestionTime). The edge label is computed once and shared by the arm's three errors:Stage 2 never produces this assignment. No production code fixes a node at
IngestionTime(only tests do, viawith_timing), and a node without a fixed timing either takes its consumer's timing or follows its ownMatAtIngestionassignment. So only hand-built or candidate-fixed plans are affected, and no candidate set changes.Key code interfaces added / modified
crates/types/src/ir/properties/execution_data_state.rsUsage, from the regression tests in
timing.rs(query_time_summary_over_unstored_ingestion_rows_is_rejected,..._state_is_rejected). Stage 2 calls the sameapply_materialization_timingsfor each materialization candidate:Before this PR
The rows DAG above validated:
apply_materialization_timingsreturnedOk. The result was a physical plan whose query-time KLL build reads rows that exist only during ingestion. The exact-state variant also passed the timing pass and failed only at export, withMaterializationTimingMismatch.After this PR
Both fail in the timing pass. The message is
a query-time SummaryAgg.child reads Aggregate output produced at ingestion time that is not stored (not MatAtIngestion), or... reads SummaryAgg output ...for the exact-state case. Storing the input node (MatAtIngestion) makes either plan valid. Stage 2 candidates, Stage 3 selections and the Example 1–4 plans and costs are unchanged.Verification
query_time_summary_over_unstored_ingestion_rows_is_rejectedandquery_time_summary_over_unstored_ingestion_state_is_rejected: each failed before its fix (left: None) and passes after.cargo nextest run -p asap-types -p asap-physical-asap-optimization -p asap-plan-selection: 435 passed.cargo nextest run -p asap-integration-testsonplanner_layering_example1..4,materialization_e2e,tumbling_pane_merge,exact_composition,reuse_rules,frequency_execution,stage2_physical_asap_optimization_sql,kll_pane_executionandprecompute_raw_samples: 119 passed. The example tests pin the selected plans and costs, so those are unchanged.asap-frontend-sqlmaintained_population+pearson_corr(15 passed),asap-executorweighted_topk_binding+promql_binary(7 passed),asap-logical-asap-optimization(178 passed).cargo fmt --all -- --checkandcargo clippy -p asap-types --all-targets --all-features --locked -- -D warnings: clean.PaneChain.childlabel on the olderQueryTimeValueUnderIngestionSummaryerrors has no dedicated test:apply_materialization_timingsrejects that shape earlier, withUpstreamNotAtIngestion.Limitations / follow-ups
PhysicalASAPDAG::validate(the exported-DAG check) has no matching rule. A hand-built or deserialized exported DAG where an unstored ingestion-time producer feeds a query-time consumer still returnsOk(()). Tracked in PhysicalASAPDAG::validate accepts an unstored ingestion-time output read at query time #747.Closes #738