Generating Transcript Version Data

Background

See Transcript Versions and Python HGVS library discussion - transcript data is distributed as gzipped JSON by our spin-off project https://github.com/SACGF/cdot/, which has loaders for both Python HGVS libraries.

You probably don't need this page. Pre-generated cdot files are published as cdot data releases, and python3 manage.py import_cdot_latest downloads and imports them for you — see Gene and Transcript Version Data. Generate your own only if you need a build/release cdot doesn't publish.

Generating from scratch

The authoritative instructions live with cdot, since that's where the tooling is: cdot - Create data from scratch.

The general shape is:

1. Download the RefSeq/Ensembl GFF/GTFs you want (all_transcripts.sh in cdot's generate_transcript_data fetches the standard set).
2. Convert each to cdot JSON.
3. Merge them.

Two different outputs are needed for the two different jobs:

• Merged / historical — many GFFs merged together, keeping the latest version of each transcript. This is what HGVS resolution uses, so it can resolve historical transcript versions.

• Single release — one GFF only. This is what a GeneAnnotationRelease is built from, so gene list filters give stable results for a given annotation version. Produced by running cdot_json.py merge_historical over a single file:

  ${CDOT_DIR}/generate_transcript_data/cdot_json.py merge_historical \
      --genome-build=GRCh38 \
      --output cdot-${CDOT_DATA}.ensembl.Homo_sapiens.GRCh38.116.gff3.json.gz \
      Homo_sapiens.GRCh38.116.gff3.json.gz
  

Which releases to pick for each build:

• Ensembl GRCh37 - has stayed on release 87
• Ensembl GRCh38 - the release matches the VEP version
• RefSeq - see the "Refseq" entry on your Annotation Version page, eg 105.20220307 - GCF_000001405.25_GRCh37.p13_genomic.gff

Then import with import_gene_annotation --release=... — see Gene and Transcript Version Data.

See also

• cdot - generate JSON.gz from GTF files, and loaders for each HGVS library
• SACGF fork of PyHGVS - legacy converter, being retired