-
Notifications
You must be signed in to change notification settings - Fork 3
Install (Server)
Setting up VariantGrid on a server, running as system services. For a development machine see Install (Developer).
The dependency installer is tested on Ubuntu 20, 22, 24 and 26 (see the header of
scripts/install/ubuntu_install_dependencies.sh). Instructions below assume a recent Ubuntu.
Create a "variantgrid" user to run the services:
sudo apt-get install git
export SYSTEM_VARIANTGRID_USER=variantgrid
sudo id -u ${SYSTEM_VARIANTGRID_USER} &>/dev/null || sudo useradd ${SYSTEM_VARIANTGRID_USER} --create-home --shell /bin/bash
sudo mkdir -p /opt/variantgrid # Or /mnt/variantgrid
sudo chown variantgrid /opt/variantgrid
sudo su variantgrid
cd /opt/
git clone https://github.com/SACGF/variantgrid variantgrid
# Installs system dependencies via apt-get (Python packages are installed separately, into a venv)
sudo variantgrid/scripts/install/ubuntu_install_dependencies.sh
VariantGrid keeps database passwords and other sensitive values out of source control, in a file it
looks for at /etc/variantgrid/settings_config.json. config/settings_config.json in the repo is a
template to start from:
sudo mkdir /etc/variantgrid
sudo cp variantgrid/config/settings_config.json /etc/variantgrid
It holds the DB password and is read by the user running VariantGrid (web + Celery), so hand ownership over to them:
sudo chown variantgrid /etc/variantgrid/settings_config.json
sudo chmod 600 /etc/variantgrid/settings_config.json
Now edit /etc/variantgrid/settings_config.json — see Settings for which values a fresh install
needs to change and how to check it parses. The database section has to match the database you create
in the common steps below.
Create the virtual environment and install the Python requirements — see Install Python venv.
Create it as the variantgrid user so the services can read it. Every python3 manage.py ... command
assumes you have activated it, and is run from the VariantGrid install dir as the variantgrid user.
Each machine gets its own settings file named after its hostname:
cd variantgrid
# Create settings file for this machine (lowercase hostname with dashes removed)
cp variantgrid/settings/env/_settings_template.py variantgrid/settings/env/$(hostname | tr '[:upper:]' '[:lower:]' | tr -d -).py
Edit it to set WEB_HOSTNAME and ALLOWED_HOSTS for this machine. The template sets secure cookies
and DEBUG = False; an intranet deployment served over plain HTTP drops the https_settings import,
as browsers never store a Secure cookie from an HTTP response.
Now do the shared part of the install — database, annotation, VEP and the upgrader: Install.
Come back here when you reach the end of that page.
You will need to modify /etc/nginx/nginx.conf to work as a proxy.
There is an example at ${VARIANTGRID_DIR}/config/nginx.conf — it runs on port 80 and handles static
files. For HTTPS, get a certificate with certbot and let it add the SSL
server block and the http → https redirect for you.
cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.ORIG
cp ${VARIANTGRID_DIR}/config/nginx.conf /etc/nginx/
Note that upload size limits are handled here via client_max_body_size (there is no Django-side
request/file size cap).
Edit the newly copied file, you need to change at least 3 lines: the IP address, and the path to the static files eg:
server {
listen 80;
server_name 130.56.244.155; # <---------------------------------- ** Change IP **
add_header Access-Control-Allow-Origin 130.56.244.155; # <--------- ** Change IP **
location /static/{
autoindex on;
root /opt/variantgrid/variantgrid/sitestatic; # <------------------ ** Change Path **
}
}
If you're on a VM with a small root partition, change where NGINX puts temp upload files (remember to assign this write permission to the nginx user - which is probably "www-data"):
client_body_temp_path /mnt/nginx_upload_temp;
For the changes to take effect you'll need to have nginx reload the config file (simply restarting nginx will not have the same effect).
sudo nginx -s reload
This is how to run VariantGrid using services, a way for Unix to run programs all the time (ie not by a user logged in on a shell). We use SystemD (every supported Ubuntu release uses it).
Create the dir for services to log to:
sudo mkdir /var/log/variantgrid
sudo chown variantgrid /var/log/variantgrid
Gunicorn and Celery rely on .env files to set various things. Copy the defaults into /etc/variantgrid:
cd ${VARIANTGRID_DIR} # Wherever this is
sudo mkdir -p /etc/variantgrid
sudo cp -r config/celery config/gunicorn config/variantgrid.env /etc/variantgrid/
/etc/variantgrid/variantgrid.env is the one file you need to look at. Every service reads it, and it holds a single setting - where VariantGrid is installed:
VG_INSTALL_DIR="/opt/variantgrid"
Set that to your install dir (eg /mnt/variantgrid) and you're done - everything else is derived from it. The services cd there before starting, so the celery/gunicorn .env files can use paths relative to it, and run out of the .venv created by uv venv in Install Python venv:
# grep GUNICORN_BIN /etc/variantgrid/gunicorn/gunicorn.env
GUNICORN_BIN=".venv/bin/gunicorn"
# grep CELERY_BIN /etc/variantgrid/celery/*.env
CELERY_BIN=".venv/bin/celery"
# grep schedule /etc/variantgrid/celery/celeryd_beat.env
CELERYD_OPTS="--detach --schedule=data/celery_beat_schedule"
If you keep your Python environment somewhere else, put an absolute path in CELERY_BIN/GUNICORN_BIN instead.
Upgrading from an older install: these files used to hardcode /usr/local/bin and the install dir, so there were per-location copies (celeryd_beat_opt.env, and config/systemd/opt vs config/systemd/mnt). Those are gone - re-copy the config as above, set VG_INSTALL_DIR, delete any leftover /etc/variantgrid/celery/celeryd_beat_opt.env, then re-install the service files (below) and sudo systemctl daemon-reload.
The service files work for any install location - they get the path from VG_INSTALL_DIR set above - so install them as-is (see also config/systemd/README.txt):
cd ${VARIANTGRID_DIR}/config/systemd
sudo cp *.service /lib/systemd/system
for i in *.service;
do sudo systemctl enable $(basename ${i});
done;
# Then start them (the scripts call "service ... start" so need root)
sudo ${VARIANTGRID_DIR}/scripts/start_services.sh
If you have to change them once they've been installed, make sure to
sudo systemctl daemon-reload
sudo ${VARIANTGRID_DIR}/scripts/restart_services.sh
for the changes to be picked up
The daemon-reload is for the .service files. The /etc/variantgrid .env files are read each time a service starts, so changing VG_INSTALL_DIR just needs sudo ${VARIANTGRID_DIR}/scripts/restart_services.sh.
To create the static folders desired by nginx, you'll need to run Django's various collect statics.
This can be achieved with scripts/upgrade.sh (as the variantgrid user) - activate the venv first, as the
script just calls python3:
sudo su variantgrid
cd ${VARIANTGRID_DIR}
source .venv/bin/activate
./scripts/upgrade.sh auto
You can create users using the admin tool, or allow people to create their own accounts via the registration process, which requires an email server. Postfix requires a domain name.
sudo apt-get install postfix
A prompt will appear. Select Internet and enter your fully qualified domain name (eg variantgrid.com)
Edit /etc/postfix/main.cf
Change the "myhostname" to be the full address (from the local VM hostname)
myhostname = variantgrid.com
Add the line:
virtual_alias_maps = hash:/etc/postfix/virtual
Check the following 2 lines are there:
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
alias_maps = hash:/etc/aliases
The 1st stops spammers and we need alias_maps to make a no-reply address.
After modifying main.cf, be sure to run service postfix reload
Run the following (be sure to adjust for your domain) to make a no-reply address
echo "devnull: /dev/null" >> /etc/aliases
echo "no-reply@variantgrid.com devnull" >> /etc/postfix/virtual
To then test sending
echo "This is the body" | mail -s "This is the subject" destinationemail@gmail.com
and check your spam folder.
To check errors:
tail /var/log/mail.err
Missing DB:
error: open database /etc/postfix/virtual.db: No such file or directory
Can be fixed via:
postmap /etc/postfix/virtual
service postfix restart
A VM root partition is usually quite small, so we need to move data storage onto mounted disks
- Run VariantGrid - Start the services and connect