spike: Design variant + annotation API shape for score set and variant pages #743

Activity

1. bencap

   added

   workstream: clinical

   app: backendTask implementation touches the backendTask implementation touches the backend

   app: frontendTask implementation touches the frontendTask implementation touches the frontend

   type: investigation

   on May 18, 2026
2. bencap

   added a parent issue on May 18, 2026

   • epic: Better Reverse Translation #746
3. bencap

   self-assigned this

   on Jun 30, 2026
4. bencap commented on Jun 30, 2026

   

   bencap

   on Jun 30, 2026

   CollaboratorAuthor

   More actions

   Spike outcome — variant + annotation API contract

   This comment is the decision record for #743: the concrete contract for the score-set/variant pages and the programmatic surface. It deliberately splits into a settled spine and a small set of explicitly-deferred items to be co-designed during the parallel UI + API rework — calling those out is part of the result, not a gap. ((D##) are decision tags carried in the fuller design notes.)

   The shape (settled)

   • Two kinds of variant, joined by Cat-VRS (D6). The assayed variant (GET /variants/{urn}, MaveDB-native, citable, version-pinned) and the mapped allele (GET /alleles/{digest}, VRS-native, deduplicated, assay-agnostic). The old conflated /mapped-variants is retired. Cat-VRS is the seam: it's anchored on the assayed side and points (by digest) into the mapped substrate — crossing is a pointer walk, not a copy.
   • One lean whole-set view backs the score-set page (D63). GET /score-sets/{urn}/variants returns a compact, pre-chewed, served-whole dataset that the table, heatmap, and baseline (score) histogram all compose from — no separate "viz lane." variant_urn is the universal selection/tooltip key; assay_level_digest is the annotation-join bridge. Cursor pagination moves to the programmatic/bulk lane.
   • Cat-VRS is spec-pure and uses a star relation model (D54). molecularRepresentation is a clean GA4GH CategoricalVariant; members are every representation, by digest; relations (coordinate_representation_of / encodes / translation_of) are member → defining, carrying score semantics only. Coordinate level is carried by level + ClinGen id, not relations. Materialized as a JSONB skeleton on MappingRecord (D33).
   • Annotations ride embedded, keyed by digest (D44–D46). No dedicated annotation endpoint yet; divergent per-member values are never collapsed to one categorical value for a protein variant (D42).
   • Temporal as_of is resource-level (D59/D60), one predicate / two modes. Supported on the single content resources (/variants/{urn}, /alleles/{digest}) — reconstruct mode for those, select mode for the immutable VA statements (§8.1). The score-set whole-set view is current-only.
   • Supersession is version selection, not a temporal query (D69). Default = current tail; superseded versions are reachable only by explicit selection (their own URN), flagged superseded_by. as_of and include_superseded are distinct, non-conflatable knobs (D73); the per-version URN makes "as_of across a supersession boundary" structurally unrepresentable.

   Answers to this issue's questions

   • Minimum sparse payload for list rendering → the lean whole-set record: variant_urn, score, a representative (measured-level) consequence, clingen_allele_id, assay_level_digest, and pre-parsed position/ref/alt at both nt and protein levels (drives the DNA/Protein heatmap toggle client-side; D70). Server-parsed — the client stops parsing/translating HGVS.
   • Expansion mechanism for full variant data → a query-param-selected variant detail panel on the score-set page today, backed by GET /variants/{urn} (the assayed variant-detail resource). (A dedicated assayed-variant page is proposed — D75.)
   • CatVRS expansion: on /variants/{urn} or a sub-resource? → directly on GET /variants/{urn} as the spec-pure molecularRepresentation, assembled on demand from the materialized skeleton. Optional lightweight GET /variants/{urn}/mapping[?as_of=] projection exists for the mapping/Cat-VRS skeleton alone.
   • Annotations inline or by reference? → inline/embedded, keyed by VRS digest, joined to the Cat-VRS members by digest (D46). Allele resources are content-addressed and CDN-cacheable (D35).
   • Response-time targets → the lean whole-set view (heaviest page load) ≤ 5–10s for the largest score sets (~648k), ~2–4 MB gzipped columnar. Single variant/allele resources bounded by construction. Validated by the O4 sizing experiment (below).
   • Variants with alleles at multiple levels → all levels are Cat-VRS members (flat, D48); the lean record carries both nt and protein parsed forms; per-codon {coding, genomic} group by shared ClinGen CA, protein by its own PA.

   Explicitly deferred — co-designed in the parallel UI + API rework (D71)

   • Annotation-histogram grain/divergence semantics (D63). The effect histogram is variant-grained and solved by the lean record. The clinical-control (ClinVar) and gnomAD histograms are allele-grained and divergent; the choice (representative pick / multi-membership / allele-grained bins) is a product decision, not an API blocker (D72) — the API serves the grain-agnostic primitive (digest-keyed per-allele values + member digests) and the grain is a client-side reduction. ClinVar is already a separate fetch today; VEP follows suit as a secondary route.
   • A dedicated assayed-variant page (D75) — the missing symmetric half of the assayed/mapped split (a citable, version-pinned landing page for urn:…#42 carrying the supersession banner). API resource exists; the page is net-new UI.

   Defects this design closes

   • Superseded data served by default. Direct-by-URN access to a superseded score set's variants/scores/counts/mapped-variants is currently unfiltered — the supersession filter lives only in the search query. The VA-Spec exports have the identical confirmed gap (annotated-variants/* in routers/score_sets.py, /va/* in routers/mapped_variant.py). The selection model (D69) + self-describing superseded_by pointer closes both. See Variant search results from superseded score sets #776.
5. bencap

   closed this as completedon Jun 30, 2026
6. bencap

   added a commit that references this issue on Jul 9, 2026

   feat(cat-vrs): build categorical variant transit on the fly from live…

   ...

   644e727
7. bencap

   added this to the R2026.3 milestone on Aug 12, 2026