Skip to content

Install VEP

Jump to bottom
Dave Lawrence edited this page Aug 26, 2026 · 35 revisions

Installing VEP

Which version to install

VEP is regularly upgraded, so treat the version numbers below as an example and check settings.ANNOTATION_VEP_VERSION (currently "116") for what your checkout expects.

The packaged settings are always the latest annotation — currently VEP 116 with columns_version 5. VEP version, columns version and the annotation data files on disk move as a set, so pick one of:

  • Latest — install VEP 116, download the current annotation data, and write no annotation settings. This is what you want for a new install, and what you upgrade to for new columns (see Annotation Column Versions for what each version added).
  • Match a database dump — if you restored Install from database dump, install the VEP version the dump was annotated with and pin your settings back to its columns_version, otherwise every variant needs re-annotating, which is the work the dump was meant to save.
  • Match an existing shared annotation drive — installs sharing a drive may be on different versions, which is why ANNOTATION_VEP_VERSION is in the path.

Whichever you choose, ANNOTATION_VEP_VERSION and columns_version have to agree — the version history and the pin_annotation_to_columns_version_N() helpers are in Annotation Column Versions.

Requirements

sudo apt-get install -y tabix mysql-server mysql-client libmysqlclient-dev perlbrew git curl libdb-dev libgd-dev pkg-config

If you don't plan on using PerlBrew - and want to use system Perl (not reccomended) then:

sudo apt-get install cpanminus

Perlbrew

Perlbrew installs Perl modules into your home directory rather than the system. I recommend it as it reduces conflicts and is easier to blow away your installation and start again.

See PerlBrew section of VEP install

su variantgrid # or whatever user you run VG as
perlbrew init # Need for next line's install to work otherwise we get 'Failed to download'
perlbrew install -j 5 --as 5.38.0 --thread --64all -Duseshrplib perl-5.38.0 --notest
perlbrew switch 5.38.0
perlbrew install-cpanm

If you use Perlbrew, set settings.ANNOTATION_VEP_PERLBREW_RUNNER_SCRIPT which will run a script to switch to appropriate perlbrew before calling VEP.

Perl dependencies

# Run after switching in Perlbrew or running as root to use system Perl

export PERL_MM_USE_DEFAULT=1 # CPAN automatic yes
cpanm Archive::Zip Archive::Extract DBD::mysql DBI Set::IntervalTree PerlIO::gzip Try::Tiny Role::Tiny::With GD Bio::Perl Test::Warnings DBD::SQLite

DO NOT install any other Perl libraries by hand. Use the versions provided by VEP.

Follow these instructions for BigWig support

Build the kent library as per the instructions above, then install the Perl module into the same Perl you'll run VEP with:

# download and build kent library as per instructions above
cpanm Bio::DB::BigFile --force # Enter kent src here (or have KENT_SRC set from install)

The Ensembl page suggests cpanm -l $HOME/cpanm Bio::DB::BigFile, which installs into a private directory. Install it into your Perlbrew Perl (as above) instead, so it is found automatically. If you do use -l, then PERL5LIB must include $HOME/cpanm/lib/perl5 everywhere VEP runs — your shell, Celery workers, and scripts/perlbrew_runner.sh — or the BigWig-based conservation columns fail at annotation time rather than at install time.

GD is the other module that often needs cpanm GD --force — its test suite fails on a TIFF test file while the module itself works.

VEP

See VEP Download/Install

VEP itself is the same either way — it's the cache that is consortium specific, and it has to match settings.ANNOTATION[build]["annotation_consortium"]. The packaged default is "RefSeq" (what Shariant and SA Pathology use); research deployments that set it to "Ensembl" need the Ensembl cache.

Because systems use shared drives, and may be on different versions, we make sure to have the VEP_VERSION explicitly in the path — ANNOTATION_VEP_VERSION is what builds ANNOTATION_VEP_CODE_DIR / ANNOTATION_VEP_PLUGINS_DIR.

Everything below lives under /data/annotation, so create it and give it to the user you run VEP and VariantGrid as before you start — see Install Annotation.

export VEP_VERSION=116  # Match settings.ANNOTATION_VEP_VERSION
export VEP_VERSION_BASE_DIR=/data/annotation/VEP/vep_code/${VEP_VERSION}
export PLUGINS_DIR=${VEP_VERSION_BASE_DIR}/plugins
mkdir -p ${PLUGINS_DIR}
cd ${VEP_VERSION_BASE_DIR}
git clone https://github.com/Ensembl/ensembl-vep.git
cd ensembl-vep
git checkout release/${VEP_VERSION}
# Install program/libraries
export VEP_CACHE=/data/annotation/VEP/vep_cache
# Do "p" and "a" as separate operations as if tests fail plugins won't be installed
perl INSTALL.pl --AUTO p --PLUGINS all --CACHEDIR ${VEP_CACHE} --PLUGINSDIR ${PLUGINS_DIR}
perl INSTALL.pl --AUTO a --PLUGINS all --CACHEDIR ${VEP_CACHE} --PLUGINSDIR ${PLUGINS_DIR}

Run INSTALL.pl from inside the ensembl-vep dir (as above) so it finds the modules it just installed.

The a (API) step finishes by running the checked-out test suite, which needs test caches you don't have yet — so expect a run of ERROR: No cache found for homo_sapiens, version <N> messages ending in Warning: Tests failed, VEP may not run correctly. The install itself is still good; verify it with the Testing commands below rather than by the test output. Add --NO_TEST to skip them.

VEP Plugins

Get the Grantham VEP Plugin (thanks to Duarte Molha) - it goes in the plugins dir with the rest:

wget -P ${PLUGINS_DIR} https://raw.githubusercontent.com/SACGF/variantgrid/refs/heads/master/annotation/annotation_data/generate_annotation/Grantham.pm

Downloading Annotation Data

If you have access to the servers, copy from:

sacgf.ersa.edu.au:/data/sacgf/reference/VEP or a server /data/annotation/VEP

Otherwise the plugin/custom data files are served as static files on variantgrid.com, mirroring the /data/annotation layout, and downloaded by:

python3 manage.py download_annotation_data

which works out the file list from your settings, so it always asks for the versions your columns_version actually wants. See Install Annotation for the URL layout, the --dry-run / --bash-script options and the md5sum manifests for verifying file contents.

The VEP cache is not part of that, and is the minimum you need. Download the one matching your VEP version and consortium, eg:

echo "VEP Cache"
export VEP_VERSION=116  # Match settings.ANNOTATION_VEP_VERSION
wget https://ftp.ensembl.org/pub/release-${VEP_VERSION}/variation/indexed_vep_cache/homo_sapiens_refseq_vep_${VEP_VERSION}_GRCh38.tar.gz
# tar xvfz homo_sapiens_refseq_vep_${VEP_VERSION}_GRCh38.tar.gz  # extract into ANNOTATION_VEP_CACHE_DIR

Drop _refseq from the filename for the Ensembl cache.

Setup Annotation Data

The annotation data has to be laid out to match the ANNOTATION setting. The easiest way to get it going is to preserve the structure used on our servers, and then overwrite ANNOTATION_VEP_BASE_DIR, as vep_config entries are relative to that dir.

The data files we host are pre-processed rather than raw — eg gnomAD ships at 500G (GRCh37) / 800G (GRCh38) and is stripped per-chromosome down to around 4G each. See Generating Annotation Data for how each file is produced, if you need to build a release yourself.

Testing

python3 manage.py vep_version --genome-build=GRCh37
python3 manage.py vep_data_check

vep_version runs VEP the same way the annotation pipeline does, so it is also the check that your Perl environment is right. If it can't find modules, the Perl it ran isn't the one you installed into: set settings.ANNOTATION_VEP_PERLBREW_RUNNER_SCRIPT (see scripts/perlbrew_runner.sh, which switches Perlbrew version before exec'ing VEP) so every caller — including Celery workers — gets the same environment.

vep_data_check reports the annotation data files that are missing or in the wrong place. Missing plugin data (eg a dbNSFP .tbi) shows up as WARNING: Failed to instantiate plugin ... from vep_version too — see Install Annotation to download it.

See also

Clone this wiki locally