Add propose_clades.py tooling to find clade designation candidates by trvrb · Pull Request #1205 · nextstrain/ncov

trvrb · 2026-06-23T19:42:12Z

Via Claude

Summary

Adds tooling to semi-automate the identification of new SARS-CoV-2 Nextstrain clade designations from open data. The design splits the work: a deterministic script gathers and surfaces evidence, and the irreducible judgement calls (which lineage to elevate, where to draw the demarcation, the clade name, the justification prose) are left to a human and/or to Claude Code. The script never edits defaults/.

#1204 is the first designation produced by running this tool with Claude Code — clade 26A (XFG.1.1). That PR is the worked example of the workflow documented here.

What's included

scripts/propose_clades.py — flags non-designated Pango lineages that have crossed the designation threshold (≥1 spike mutation vs the parent clade and >30% regional or >20% global frequency) and writes a ranked candidate dossier to results/clade_candidates.{md,json}. Data sources, all open:
- the Nextclade SARS-CoV-2 reference tree for clean lineage-defining mutations and parent-clade structure (the ncov build is not cleanly monophyletic by Pango lineage, so it is not used for mutations);
- the Nextstrain open 6m builds (global / north-america / europe) tip-frequencies for the headline frequency, aggregated by Nextclade_pango label;
- the LAPIS open API for exact per-lineage regional frequency;
- the forecasts-ncov open MLR results for growth advantage (fitness) and modeled frequency.
It keeps a strict 1:1 Pango↔clade mapping, clusters nested candidates as alternative demarcations and ranks them by fitness differential to support the "where to draw the line" decision, annotates each nuc row with the spike amino-acid change it causes, emits paste-ready edits for all five defaults/ files, and suggests the next clade name using the designation (current) year. It also reads existing designations from defaults/, so it correctly anchors candidates to locally-designated clades the reference tree does not yet encode.
.claude/skills/designate-clades/SKILL.md — a Claude Code skill documenting the end-to-end loop: run the script, cross-check the dossier against the live resources, decide the demarcation, write the five defaults/ edits, validate with expand-clade-definitions, and draft the PR in the established style.
docs/dev_docs.md — a new "Proposing new clade designations" section covering the criteria, how to run the script, the intended Claude Code path, and how a designation PR is made.

How it behaves

Because open data is sparse and ~2 months lagged, the dossier shows each frequency alongside its source (build tip-frequency, LAPIS, MLR) so the numbers can be triangulated and cross-checked against the live resources rather than trusted blindly. The script is read-only with respect to the repo and only writes to the (git-ignored) results/ directory.

Test plan

python3 scripts/propose_clades.py runs against the local open global build and writes results/clade_candidates.{md,json}
Flagged candidates reproduce known recent designations and the demarcation tables/fitness numbers are sane
A generated clades.tsv block expands cleanly through scripts/expand-clade-definitions (augur)
After a designation is added to defaults/, re-running excludes that lineage and re-anchors its descendants to the new clade

🤖 Generated with Claude Code

trvrb · 2026-06-23T19:53:04Z

Here's the resulting Markdown output of running this script on current master

clade_candidates.md

trvrb · 2026-06-26T23:08:57Z

I spent more time with this working from local GISAID data. I'm pretty happy with how it's performing. A review of current diversity just flagged XFG.3.4.1 as a completely obvious designation. I made a PR for it here #1206. Other things are not quite so obvious at the moment. See attached Markdown dossier.

clade_candidates.md

corneliusroemer · 2026-06-29T15:15:52Z

+
+A candidate Pango lineage qualifies when it **both**:
+1. carries **>=1 spike (S:) mutation** relative to its parent Nextstrain clade, and
+2. has risen to **>30% regional** (North America or Europe) **or >20% global** frequency.


We also have:

Suggested change

2. has risen to **>30% regional** (North America or Europe) **or >20% global** frequency.

2. has risen to **>30% regional** (North America or Europe) **or >20% global** frequency or shows consistent >0.05 per day growth in frequency where it's circulating and has reached >5% regional frequency.

https://nextstrain.org/blog/2022-04-29-SARS-CoV-2-clade-naming-2022

corneliusroemer · 2026-06-29T15:17:03Z

+
+Designating a new Nextstrain clade is a judgement call on top of a quantitative threshold:
+a candidate Pango lineage should carry **≥1 spike mutation** relative to its parent clade and
+have risen to **>30% regional** or **>20% global** frequency (see the


Suggested change

have risen to **>30% regional** or **>20% global** frequency (see the

have risen to **>30% regional** or **>20% global** frequency or shows consistent >0.05 per day growth in frequency where it's circulating and has reached >5% regional frequency (see the

corneliusroemer · 2026-06-29T15:17:47Z

+### Intended path for Claude Code
+
+This tool is designed to be driven by [Claude Code](https://claude.com/claude-code) via the
+`designate-clades` skill (`.claude/skills/designate-clades/SKILL.md`). The skill walks through the
+full loop:
+
+1. Run `scripts/propose_clades.py` and read `results/clade_candidates.md`.
+2. For each flagged candidate, cross-check against the live resources (the open builds at
+   `nextstrain.org/ncov/open/{global,north-america,europe}/6m`, the Cov-Spectrum lineage page,
+   forecasts-ncov, and the Nextclade reference tree).
+3. Decide the demarcation: among nested candidate lineages, elevate the one where the MLR growth
+   advantage jumps most and frequency is rising; or decline if it is just slicing the current
+   dominant clade finer.
+4. Write the edits (see below) and validate them.
+5. Draft the PR.
+
+Open the skill in a Claude Code session in this repo and ask it to find or designate new clades.


Maybe move to AGENTS.md or CLAUDE.md

xz-keg · 2026-07-25T15:32:25Z

API for local gisaid+genbase seqs (for seqs 2025-6-1 and later)

https://cpredict.cn/spectrum/claude.md

Add tooling to semi-automate identification of new SARS-CoV-2 Nextstrain clade designations from open (GenBank/INSDC) data, motivated by the loss of GISAID API access. The script gathers and surfaces evidence; the judgement calls (which lineage to elevate, where to demarcate, the name, the prose) are left to a human or Claude Code. scripts/propose_clades.py flags non-designated Pango lineages that have crossed the designation threshold (>=1 spike mutation vs the parent clade and >30% regional or >20% global frequency) and writes a ranked candidate dossier to results/clade_candidates.{md,json}. It never edits defaults/. Data sources, all open: - Nextclade SARS-CoV-2 reference tree for clean lineage-defining mutations and parent-clade structure (the ncov build is not cleanly monophyletic by Pango lineage, so it is not used for mutations); - Nextstrain open 6m builds (global/north-america/europe) tip-frequencies for the headline frequency, aggregated by Nextclade_pango label; - LAPIS open API for exact per-lineage regional frequency; - forecasts-ncov open MLR for growth advantage (fitness) and modeled frequency. The tool keeps a strict 1:1 Pango-to-clade mapping, clusters nested candidates as alternative demarcations and ranks them by fitness differential to support the "where to draw the line" decision, annotates each nuc row with the spike amino-acid change it causes, emits paste-ready edits for all five defaults/ files, and suggests the next clade name using the designation (current) year. It reads existing designations from defaults/ so it correctly anchors candidates to locally-designated clades the reference tree does not yet encode. Add .claude/skills/designate-clades/SKILL.md documenting the end-to-end loop for Claude Code: run the script, cross-check the dossier against live resources, decide the demarcation, write the five defaults/ edits, validate with expand-clade-definitions, and draft the PR in the established style. Document the tool, the intended Claude Code path, and how designation PRs are made under a new section in docs/dev_docs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add a --data-source {open,gisaid} affordance so the tool can evaluate either GISAID-free open data or a freshly built+uploaded GISAID set, which gives far better global/Europe/Asia coverage and surfaces sweeps the North-America-dominated open data misses. Fetching is unified through nextstrain.org charon getDataset(prefix) (+ the tip-frequencies sidecar); only the dataset prefixes differ by source, and every build is now fetched live rather than read from a local file: - open: the live /ncov/open/{global,north-america,europe}/6m core builds, plus the LAPIS open API and the open forecasts-ncov MLR endpoint. - gisaid: global + all 6 regions, auto-discovered from the group's dated Groups uploads via charon getAvailable (groups/<group>/ncov/gisaid/<region>/6m/<date>). The latest upload date is chosen automatically (--group, --gisaid-date to pin, --gisaid-url-template to override); the GISAID MLR endpoint is used and LAPIS is skipped (no API access). Region frequencies are now region-filtered. The regional builds are subsampled focal+contextual (e.g. the Asia build is only ~61% Asian tips), so a region's frequency is computed over only the tips whose `region` attribute matches it — the ?f_region=<Region> equivalent — via Build.current_freq(members, region=...). Global frequency stays unfiltered. This also fixes a latent bias in the previous open NA/Europe handling, which averaged over focal+contextual tips. Flagging generalizes from the hardcoded NA/Europe pair to "global > 20% OR any region > 30%", and the dossier renders whatever region set was analyzed (header notes the source and, for gisaid, the discovered date). Add an effective-sample-size (Kish) warning. Sequencing is sparse and lagged, so a region's latest-pivot KDE estimate can rest on very few sequences (open Europe ~6; gisaid africa ~2, asia ~6, europe ~12). The dossier now prints a per-region `low-n=<n>` warning from Build.region_effective_n() so thin regions are visibly unreliable — important for gisaid, which has no LAPIS backstop. Update docs/dev_docs.md and the designate-clades skill to document the open vs gisaid runs, auto-discovery, region-filtering, and the low-n caveat. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

… pivot The >30% regional / >20% global threshold was read from a single number per geography: the KDE tip-frequency at the build's latest pivot. That pivot rests on a handful of the most-recent, extrapolated sequences (effective n we measured: open Europe ~6, gisaid africa ~2), so it produced noise-driven flags — e.g. "RR.1 Europe 69%" (8 of ~13 tips; LAPIS says 2%) and "BA.3.2.2 Africa 100%" (~2 tips). Clades are a robust coarse-graining of majority trends (Pango already flags handfuls), so the call should be conservative and backward-looking. Reframe the threshold as a one-sided test on a proportion and judge the lower confidence bound, not the point estimate. Per candidate, per geography (global = global build unfiltered; each region = its build filtered to region==<Region>): - Band: build tips with collection date in [now-6mo, now-3wk] (the lower bound drops the 2020-2026 context backbone the "6m" builds carry; the upper bound is a reporting-lag guard). "now" = the global build's latest pivot. - Slide a fixed-N=50 window of date-sorted tips. Fixing the sample size (not the time-width) gives every window comparable CI width, so a real peak is never split across two under-powered windows; sparse periods just yield wider-in-time windows. Counts come from the build subsample, which is population-weighted (group_by country/month + population_weights) and is the object historical clade frequencies use -- raw GISAID counts are country-biased (recent "Asia" is ~54% Singapore, capped to 6 in the build). - Flag the geography iff the one-sided 95% Wilson lower bound clears the threshold in some window, with a rough Bonferroni over the effective-independent windows (m ~ band/4wk ~ 6, z ~ 2.39). A flag therefore means *confidently* over the line (effective point-estimate bar ~46% regional / ~34% global at N=50, a documented knob), not a noisy point estimate. - A geography with < 50 band tips reports "insufficient" and never flags -- the principled too-little-data cutoff. The dossier now reports each geography's peak window (peak X% [n, as of <date>, lo Y%]) and current window, so a lineage dominant months ago still surfaces with its trajectory, and the demarcation table follows suit. LAPIS stays as open-only secondary corroboration; it no longer drives the flag. Implementation: new wilson_lower(), Build.band_tips(), windowed_assessment(); build_candidate / demarcation_chain / renderers reworked; the dead latest-pivot KDE paths (Build.trajectory/current_freq, region_effective_n, per-tip freqs) removed. New knobs --window-n / --lookback-months / --lag-days. docs/dev_docs.md and the designate-clades skill document the method, the effective bar, and "insufficient". Verified: the old noise flags die (open RR.1 drops out entirely; gisaid BA.3.2.2 no longer flags on Africa's ~2 tips) while genuine sustained majorities still flag with n=50 windows whose lower bound clears the threshold (BA.3.2.2 Asia 64%/lo47%, Europe 48%/lo32%; XFG.1.1 North America 48%/lo32%), each with a sane "as of" date. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…ion diffs Two additive dossier enhancements for reviewing flagged candidates: - Verify-in-Auspice links: for each candidate, emit one live nextstrain.org link per geography that cleared the threshold, coloring the tree by the candidate's defining nuc genotypes (c=gt-nuc_<sites>) and filtering to that region (f_region=<Region>, omitted for global). The dataset URL is derived from the geography's resolved prefix, so it works for both data sources (open /ncov/open/<region>/6m and gisaid /groups/<group>/ncov/gisaid/<region>/6m/<date>). Lets the reviewer eyeball the clade's prevalence directly instead of trusting the number; links land in both the .md and the .json. - Alternative-demarcation mutation diffs: when nested lineages compete for one clade name, the dossier now spells out how they differ — the mutations each carries beyond the core shared by all of them (broadest cut first, spike changes annotated), instead of just listing the lineage names. This also surfaces sibling-vs-nested structure (e.g. RE.1 and RE.2 each add a different set to BA.3.2.2). Both are additive rendering off existing record fields (nuc_rows, the per-geography windowed assessments); no change to flagging or the clades.tsv block. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Refinements surfaced by the 26B (XFG.3.4.1) designation: - Default to ALL defining nuc mutations, not just the spike-linked subset, so a clade is exactly specific to its Pango lineage and demarcates precisely at it rather than a broader/higher node. Updates the skill's Step 4 and the dossier text in propose_clades.py (the "trim to a characteristic subset" caption and the per-candidate hint become "use all rows" / informational spike flagging). Text only — the script already emitted the full net set. - Add a skill step to build the prominent region(s) locally and `nextstrain deploy` a branch-named dataset to s3://nextstrain-staging, linked in the PR — the standard practice for PR-related ephemera (live at staging/ncov/<branch>/<region>/6m). - Expand the PR-drafting step to require a thorough per-region frequency breakdown from the windowed dossier (peak %, n=50 one-sided 95% lower bound, as-of date, current %, plus global) and the staging link, rather than a one-line summary. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Lets a run drop one or more regions (e.g. --skip-regions oceania) so the dossier isn't dominated by a region of less interest. The named regions are removed from the prefix set before fetching, so they're not downloaded, analyzed, or rendered, and their region-only sweeps no longer surface. Input is case/space-forgiving (maps "Oceania" / "South America" to the slug), global cannot be skipped, and an unknown region warns rather than fails. Complements --skip-regional (global only). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

FedeGueli · 2026-08-15T06:46:34Z

Very interesting step forward i think PQ.16.1.1, RF.5 RW.1.1 SW.2.1 could be the next ones to match the criteria now with RK.1 and RV.1 also above the threshold

trvrb changed the title ~~Add propose_clades.py: tooling to find clade-designation candidates from open data~~ Add propose_clades.py tooling to find clade designation candidates Jun 23, 2026

trvrb requested review from corneliusroemer and rneher June 23, 2026 19:50

trvrb force-pushed the propose-clades branch from 12eddd0 to 009ab7c Compare June 26, 2026 20:09

corneliusroemer reviewed Jun 29, 2026

View reviewed changes

trvrb force-pushed the propose-clades branch from 2ef851b to 62003e8 Compare July 6, 2026 23:41

trvrb and others added 6 commits August 6, 2026 08:11

trvrb force-pushed the propose-clades branch from 62003e8 to 16fd447 Compare August 6, 2026 06:12

Navigation Menu

Uh oh!

Add propose_clades.py tooling to find clade designation candidates - #1205

Add propose_clades.py tooling to find clade designation candidates#1205
trvrb wants to merge 6 commits into
masterfrom
propose-clades

trvrb commented Jun 23, 2026 •

edited

Loading

Uh oh!

trvrb commented Jun 23, 2026

Uh oh!

trvrb commented Jun 26, 2026

Uh oh!

corneliusroemer Jun 29, 2026

Uh oh!

corneliusroemer Jun 29, 2026

Uh oh!

corneliusroemer Jun 29, 2026

Uh oh!

xz-keg commented Jul 25, 2026

Uh oh!

FedeGueli commented Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	2. has risen to >30% regional (North America or Europe) or >20% global frequency.
	2. has risen to >30% regional (North America or Europe) or >20% global frequency or shows consistent >0.05 per day growth in frequency where it's circulating and has reached >5% regional frequency.

	have risen to >30% regional or >20% global frequency (see the
	have risen to >30% regional or >20% global frequency or shows consistent >0.05 per day growth in frequency where it's circulating and has reached >5% regional frequency (see the

Uh oh!

Conversation

trvrb commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's included

How it behaves

Test plan

Uh oh!

trvrb commented Jun 23, 2026

Uh oh!

trvrb commented Jun 26, 2026

Uh oh!

corneliusroemer Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

corneliusroemer Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

corneliusroemer Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

xz-keg commented Jul 25, 2026

Uh oh!

FedeGueli commented Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

trvrb commented Jun 23, 2026 •

edited

Loading