|
| 1 | +# Upgrading to EQL 3.1 |
| 2 | + |
| 3 | +This guide covers behaviour in 3.1 that callers should know about. Nothing in |
| 4 | +it requires you to change any SQL. |
| 5 | + |
| 6 | +## TL;DR |
| 7 | + |
| 8 | +1. Ordered comparisons of block-ORE text values now check every paired term, |
| 9 | + so a malformed term anywhere in a stored value raises, where before it |
| 10 | + raised only if the comparison reached it. Check your `_ore` text columns |
| 11 | + before upgrading — |
| 12 | + [U-001](#u-001-block-ore-text-comparisons-check-every-term). |
| 13 | + |
| 14 | +## Compatibility |
| 15 | + |
| 16 | +| Surface | Status | |
| 17 | +| --- | --- | |
| 18 | +| Ordering results of every block-ORE operator and the ORE operator class | **Unchanged** for well-formed values. | |
| 19 | +| Block-ORE text values (`public.eql_v3_text_ord_ore`, `public.eql_v3_text_search_ore`) with a malformed or mismatched-width term | **Changed.** Every comparison against the value raises — see U-001. | |
| 20 | +| Wire format, domains, index terms | **Unchanged.** No re-encryption, no data migration. | |
| 21 | + |
| 22 | +## Upgrade notes |
| 23 | + |
| 24 | +### U-001: Block-ORE text comparisons check every term |
| 25 | + |
| 26 | +**What changed.** A block-ORE text value carries up to six ORE terms. The |
| 27 | +comparator behind its ordered operators (`<`, `<=`, `>`, `>=`), `ORDER BY` and |
| 28 | +the ORE operator class used to compare term by term and stop at the first |
| 29 | +unequal pair. That made its run time reveal how long a prefix two values |
| 30 | +share, so it now compares every paired term in a constant number of |
| 31 | +operations. A consequence is that every paired term is now checked. A term |
| 32 | +that is malformed (its length is not 49 × N + 16 bytes for some N ≥ 1) or |
| 33 | +whose width differs from the term it is paired with raises: |
| 34 | + |
| 35 | +``` |
| 36 | +ERROR: Malformed ORE term: 407 bytes |
| 37 | +ERROR: Ciphertexts are different lengths |
| 38 | +``` |
| 39 | + |
| 40 | +Before 3.1 the comparator raised only when it reached the bad term, which it |
| 41 | +did not when an earlier term already settled the order, or when the bad term |
| 42 | +was paired with a NULL element. |
| 43 | + |
| 44 | +**Who is affected.** Only rows whose `ob` terms were not produced by a |
| 45 | +CipherStash client: hand-edited payloads, truncated values, or payloads from |
| 46 | +another encoder. Values encrypted by CipherStash clients are well formed, and |
| 47 | +single-term domains (every block-ORE domain but text) are unaffected. |
| 48 | + |
| 49 | +**Why it matters.** One bad row can now fail a whole query that orders or |
| 50 | +range-scans the column, and `CREATE INDEX` or `REINDEX` on it, instead of |
| 51 | +being skipped silently. |
| 52 | + |
| 53 | +**What to do.** Before upgrading, list the term widths in each block-ORE text |
| 54 | +column. Every term in a column should have the same width: |
| 55 | + |
| 56 | +```sql |
| 57 | +SELECT octet_length(term.bytes) AS bytes, |
| 58 | + (octet_length(term.bytes) > 16 |
| 59 | + AND (octet_length(term.bytes) - 16) % 49 = 0) AS well_formed, |
| 60 | + count(*) AS terms |
| 61 | +FROM my_table t, |
| 62 | + unnest((eql_v3_internal.ore_block_256(t.my_column::jsonb)).terms) AS term |
| 63 | +WHERE term.bytes IS NOT NULL |
| 64 | +GROUP BY 1, 2 |
| 65 | +ORDER BY 3 DESC; |
| 66 | +``` |
| 67 | + |
| 68 | +One row, with `well_formed` true, means the column is clean. For any other |
| 69 | +width, find the rows that carry it and re-encrypt them: |
| 70 | + |
| 71 | +```sql |
| 72 | +SELECT t.ctid |
| 73 | +FROM my_table t |
| 74 | +WHERE EXISTS ( |
| 75 | + SELECT 1 |
| 76 | + FROM unnest((eql_v3_internal.ore_block_256(t.my_column::jsonb)).terms) AS term |
| 77 | + WHERE octet_length(term.bytes) <> 408 -- the width the first query reports |
| 78 | +); |
| 79 | +``` |
| 80 | + |
| 81 | +## Rollback |
| 82 | + |
| 83 | +Installing the 3.0 bundle restores the term-by-term comparator. Applying any |
| 84 | +EQL bundle drops and recreates its schemas, so re-apply grants and dependent |
| 85 | +objects afterwards — see |
| 86 | +[U-002 in the 3.0.5 guide](v3.0.5.md#u-002-grants-and-dependent-objects-do-not-survive-an-install). |
| 87 | + |
| 88 | +## See also |
| 89 | + |
| 90 | +- [`tests/timing/ore_block_256/`](../../tests/timing/ore_block_256/README.md): |
| 91 | + the timing harness behind U-001, and its results. |
0 commit comments