MaveDB: only keep the columns and records that can be used

[davmlaw]

Fixes the memory usage reported in #852, where a single 229 bp deletion makes the plugin allocate
9.7 GB

Two independent changes, both bounded by the variant's reference span rather than by anything the user
controls:

Keep only the columns parse_data needs. It built %res keyed on every column in the data file
(259 in the current distribution) for every record in the span, and _join_results narrowed to cols
afterwards — so cols never reduced the memory. _set_parsed_columns() works the wanted indexes out
once in new(). It keeps MaveDB_hgvs and MaveDB_refseq beyond cols, since run() reads those
directly, and only when the option that consults them is enabled.

Skip records that cannot match. Score sets mapped through VRS carry a ga4gh:VA.<digest> in ref
instead of sequence — 77% of records in the current file. get_matched_variant_alleles compares alleles
as strings, so those can never match; they were parsed, hashed and held anyway.

On the 229 bp deletion from #852, against the current MaveDB_variants.tsv.gz:

	peak RSS	wall
before	9.68 GB	19.7 s
columns only	0.71 GB	9.7 s
record skip only	0.08 GB	3.6 s
this PR	0.08 GB	3.6 s

0.08 GB is what the same run costs with the plugin not loaded at all.

Output is unchanged

Compared line by line against the current plugin, identical in every case:

• t/testdata/cache (chr21, GRCh38) with the chr21 slice of the MaveDB file, over 352 variants from
  t/testdata/input/test.vcf plus records inside the cache window — 2,112 MaveDB values from both,
  0.20 GB → 0.10 GB.
• 824 variants sampled genome-wide from the MaveDB file — 31,441 MaveDB values from both,
  0.52 GB → 0.35 GB.
• Every combination of single_aminoacid_changes / transcript_match (0/0, 0/1, 1/0, 1/1), and
  cols=all. cols=all is the useful case: it makes the projection a no-op and exercises the record
  skip on its own.

Targeting main per CONTRIBUTING.md.

[davmlaw]

One clarification on the table in the description, since it invites the wrong conclusion: on that 229 bp
deletion the record skip alone already reaches the floor, which makes the column projection look
redundant. It isn't — that variant is at one extreme, where 100% of the records fetched are VRS-mapped.

Measured on VEP's own shipped test data (t/testdata/cache, chr21) there is a case at the other
extreme: a 1 kb deletion whose span contains no VRS-mapped records at all, where the skip does nothing
and the projection provides the entire saving (0.13 GB -> 0.08 GB).

Full numbers and a reproduction recipe: #852 (comment)
#852 (comment)

[ainefairbrother]

Hi @davmlaw,

Thank you for this fix. I have tested it and see that, indeed, it does speed up the plugin.

[davmlaw]

Cool. It was mostly the RAM use for me (I'm running a few VEPs in parallel on a VM) but faster is good too.

I think the Mave plugin was fine for the initial version of the data, but MAVE has grown a huge amount over the last few years (which is great!) just need a few adjustments to keep up