Repository navigation
perf(tenant): distribute the project clause into every ownership arm - #26
Merged
Merged
Conversation
An index can serve an "$or" only when every arm applies all of its bounds
at index level. The project narrowing sat on top of the canonical-first
"$or" as a separate conjunct, which made it a residual the planner had to
resolve per candidate document — collapsing a paged, sorted read into a
full scan of the tenant's collection followed by a blocking sort.
Push the project clause into each ownership arm instead. This is the
distributive law, not a policy change: {(A or B) and P} and {(A and P) or
(B and P)} select identical documents.
Two supporting shape changes make the arms index-resolvable:
- MissingCanonicalOrganisation is now a single-key $in over null and ""
instead of an $or with an $exists arm. A $exists:false test cannot be
answered from a standard index — a missing field and an explicit null
are the same index entry — while a null test already matches a missing
field. Same documents, two point bounds.
- The legacy ownership arm is a flat conjunction rather than a nested
"$and", so its bounds sit directly on the arm.
Both merge steps guard against a field constrained by both halves and fall
back to the conjunctive shape when they see one: slower, but never wrong.
ScopeWithProject now always returns a top-level "$or" rather than an "$and"
when a project is selected. All five hub-api call sites (videowalls, groups,
sites, media, labels) wrap the result in their own "$and" and none assigns
into the returned map, so the flip is safe; the doc comment says so for
future callers.
Measured on MongoDB 8.0.4 over 120k media documents using only the indexes
production already carries: owner media page 1 goes from 72,000 documents
examined / 1314 ms (SORT <- FETCH <- OR) to 31 documents / 11 ms
(SUBPLAN <- LIMIT <- SORT_MERGE). Result sets were compared as _id sets,
not counts, and are identical.
Caveats, honestly: this has not been validated on AWS DocumentDB, whose
planner treats "$or" differently. The device-restricted sub-user path does
not improve — there is no index on the legacy owner field. Neither does
page 2, because hub-api appends its keyset cursor as a second top-level
residual; a probe shows distributing the range half of that cursor fixes it,
but that is a hub-api change and is left as follow-up.
Also bumps models to v1.7.22, which carries the matching index-bounded form
of ProjectScopeFilter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change restructures tenant and project scoping so MongoDB can fully resolve ownership queries from indexes instead of degrading into collection scans.
Previously, project filters were applied as a top-level conjunction on top of the ownership
$or. While logically correct, that query shape forced MongoDB to treat the project clause as a residual filter, preventing efficient index merges across ownership branches. As datasets grow, this turns paged tenant reads into expensive collection scans followed by blocking sorts.This PR distributes the project clause into every ownership arm, preserving the same semantics while making every branch independently index-resolvable. The result is significantly more efficient tenant-scoped reads without changing access rules or widening scope boundaries.
The change also simplifies migration predicates from
$or+$existschecks into point-matchable$inexpressions. This keeps legacy ownership detection index-friendly and avoids planner fallbacks caused by non-indexable existence checks.Additional safeguards and tests were added to ensure:
The dependency update pulls in the model-layer support required for the new project scope behavior.