Skip to content

Classification: fix N+1 queries in the allele groupings grid and tasks #1729 - #1731

Open
davmlaw wants to merge 1 commit into
masterfrom
issue_1729_n_plus_one_classification
Open

Classification: fix N+1 queries in the allele groupings grid and tasks #1729#1731
davmlaw wants to merge 1 commit into
masterfrom
issue_1729_n_plus_one_classification

Conversation

@davmlaw

@davmlaw davmlaw commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

🤖 Written by Claude.

Addresses part of #1729 — the classification fixes that are just select_related/prefetch_related or hoisting a query out of a loop. Split out of #1727; the snpdb/annotation half is in a separate PR.

Allele Groupings grid

The worst of them. render_allele runs a ClassificationGrouping query per row, then c_hgvs_for walked latest_classification_modification -> classification -> allele_info -> allele_info[gb] -> ri.genome_build as lazy FK loads for every grouping of every row.

ClassificationGrouping.update() sets latest_allele_info from best_classification.classification.allele_info in the same breath as latest_classification_modification, so the stored field is exactly what that chain resolves to — read it directly and select_related the resolved variant info for both builds.

The Labs column had the same problem from the other end: allele_origin_dict prefetched classificationgrouping_set but not lab, and sorted(labs) calls Lab.__lt__, which reaches for organization. Both now come through the prefetch.

DiscordanceReport.update()

Runs on every classification publish/withdraw touching a discordant clinical context. Two problems: classification_original.classification was dereferenced per row (two queries each, one of them the wide ClassificationModification), and each newly added classification was fetched with its own .get(). Now one select_related queryset for the existing rows and one bulk query for the added ones.

Behaviour note for review: the old .get(is_last_published=True, classification=vcm_id) raised DoesNotExist if a classification had no published modification, failing the whole update. The bulk query skips it instead. I think skipping is the better behaviour, but it is a change, so flagging it.

Others

  • update_all_dirtyupdate() dirties allele_origin_grouping immediately, so it is worth bringing along; this runs over tens of thousands of groupings after a bulk import.
  • The nightly ClinVar prepare ran two COUNT queries per allele; collapsed into one aggregate with a filtered Count.

Not included

The remaining classification grid items in the issue (imported_allele_info_view, clinvar_export_view) need the pre_render(qs) hook to bulk-load a page's objects into a dict. That is a consistent little pattern rather than a one-line change, so it is better as its own PR than bolted onto this one.

Testing

python3 manage.py test --keepdb classification.tests — 98 tests pass.

…#1729

The allele groupings grid walked latest_classification_modification ->
classification -> allele_info as lazy FK loads per grouping per row.
ClassificationGrouping.update() already stores that as latest_allele_info, so
read it directly and select_related the resolved builds. The labs column
prefetched classificationgrouping_set but not lab, and sorting labs touches
organization, so pull those through too.

DiscordanceReport.update() dereferenced classification_original.classification
per row, then fetched each newly added modification one at a time - now one
query for the lot. Note this skips a classification with no is_last_published
modification where the previous get() would have raised.

Also: update_all_dirty allele_origin_grouping, and the nightly ClinVar prepare
running two counts per allele.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant