Skip to content

Commit 9d5430e

Browse files
committed
docs(eql): minor bump and an upgrade note for the comparator's new errors
Multi-term ORE values now raise on a malformed or mismatched-width term the comparator used to skip. That is a behaviour change, so the changeset is minor, and U-001 in a new 3.1 upgrade guide shows how to find such rows before upgrading.
1 parent ec9c3de commit 9d5430e

2 files changed

Lines changed: 107 additions & 10 deletions

File tree

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,21 @@
11
---
2-
'@cipherstash/eql': patch
2+
'@cipherstash/eql': minor
33
---
44

55
Security: the ORE block comparator (`eql_v3_internal.compare_ore_block_256_term`
66
and `compare_ore_block_256_terms`, behind every ordered ORE operator and the ORE
7-
operator class) now runs a constant number of operations whatever its operands.
8-
Previously its run time depended on where two values first differ, so a party
9-
able to time queries could learn how long a prefix their query value shares with
10-
stored values. For text the effect was large: a comparison took about 11 µs when
11-
the first term differs and 47 µs when only the sixth does. Results are unchanged.
12-
Single-term comparisons cost about 6% more; text comparisons now take about
13-
18 µs whatever the operands. Multi-term (text) values now check every paired
14-
term, so a malformed term after the first difference, or one paired with a NULL
15-
element, raises instead of being skipped. A timing harness is in `packages/eql/tests/timing/ore_block_256/`.
7+
operator class) now runs a constant number of operations for operands of the
8+
same shape (term count, term width, NULL positions), whether and wherever they
9+
differ. Previously its run time depended on where two values first differ, so a
10+
party able to time queries could learn how long a prefix their query value
11+
shares with stored values. For text the effect was large: a comparison took
12+
about 11 µs when the first term differs and 47 µs when only the sixth does.
13+
Results are unchanged. Single-term comparisons cost about 6% more when the
14+
terms differ and about 36% more (roughly 1 µs) when they are equal, since equal
15+
terms now run the hash too; text comparisons now take about 18 µs whatever the
16+
operands. Multi-term (text) values now check every paired term, so a
17+
malformed or mismatched-width term after the first difference, or one paired
18+
with a NULL element, raises instead of being skipped; see
19+
[U-001](../../docs/upgrading/v3.1.md#u-001-block-ore-text-comparisons-check-every-term)
20+
for how to find such rows before upgrading. A timing harness is in
21+
`packages/eql/tests/timing/ore_block_256/`.
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Upgrading to EQL 3.1
2+
3+
This guide covers behaviour in 3.1 that callers should know about. Nothing in
4+
it requires you to change any SQL.
5+
6+
## TL;DR
7+
8+
1. Ordered comparisons of block-ORE text values now check every paired term,
9+
so a malformed term anywhere in a stored value raises, where before it
10+
raised only if the comparison reached it. Check your `_ore` text columns
11+
before upgrading —
12+
[U-001](#u-001-block-ore-text-comparisons-check-every-term).
13+
14+
## Compatibility
15+
16+
| Surface | Status |
17+
| --- | --- |
18+
| Ordering results of every block-ORE operator and the ORE operator class | **Unchanged** for well-formed values. |
19+
| Block-ORE text values (`public.eql_v3_text_ord_ore`, `public.eql_v3_text_search_ore`) with a malformed or mismatched-width term | **Changed.** Every comparison against the value raises — see U-001. |
20+
| Wire format, domains, index terms | **Unchanged.** No re-encryption, no data migration. |
21+
22+
## Upgrade notes
23+
24+
### U-001: Block-ORE text comparisons check every term
25+
26+
**What changed.** A block-ORE text value carries up to six ORE terms. The
27+
comparator behind its ordered operators (`<`, `<=`, `>`, `>=`), `ORDER BY` and
28+
the ORE operator class used to compare term by term and stop at the first
29+
unequal pair. That made its run time reveal how long a prefix two values
30+
share, so it now compares every paired term in a constant number of
31+
operations. A consequence is that every paired term is now checked. A term
32+
that is malformed (its length is not 49 × N + 16 bytes for some N ≥ 1) or
33+
whose width differs from the term it is paired with raises:
34+
35+
```
36+
ERROR: Malformed ORE term: 407 bytes
37+
ERROR: Ciphertexts are different lengths
38+
```
39+
40+
Before 3.1 the comparator raised only when it reached the bad term, which it
41+
did not when an earlier term already settled the order, or when the bad term
42+
was paired with a NULL element.
43+
44+
**Who is affected.** Only rows whose `ob` terms were not produced by a
45+
CipherStash client: hand-edited payloads, truncated values, or payloads from
46+
another encoder. Values encrypted by CipherStash clients are well formed, and
47+
single-term domains (every block-ORE domain but text) are unaffected.
48+
49+
**Why it matters.** One bad row can now fail a whole query that orders or
50+
range-scans the column, and `CREATE INDEX` or `REINDEX` on it, instead of
51+
being skipped silently.
52+
53+
**What to do.** Before upgrading, list the term widths in each block-ORE text
54+
column. Every term in a column should have the same width:
55+
56+
```sql
57+
SELECT octet_length(term.bytes) AS bytes,
58+
(octet_length(term.bytes) > 16
59+
AND (octet_length(term.bytes) - 16) % 49 = 0) AS well_formed,
60+
count(*) AS terms
61+
FROM my_table t,
62+
unnest((eql_v3_internal.ore_block_256(t.my_column::jsonb)).terms) AS term
63+
WHERE term.bytes IS NOT NULL
64+
GROUP BY 1, 2
65+
ORDER BY 3 DESC;
66+
```
67+
68+
One row, with `well_formed` true, means the column is clean. For any other
69+
width, find the rows that carry it and re-encrypt them:
70+
71+
```sql
72+
SELECT t.ctid
73+
FROM my_table t
74+
WHERE EXISTS (
75+
SELECT 1
76+
FROM unnest((eql_v3_internal.ore_block_256(t.my_column::jsonb)).terms) AS term
77+
WHERE octet_length(term.bytes) <> 408 -- the width the first query reports
78+
);
79+
```
80+
81+
## Rollback
82+
83+
Installing the 3.0 bundle restores the term-by-term comparator. Applying any
84+
EQL bundle drops and recreates its schemas, so re-apply grants and dependent
85+
objects afterwards — see
86+
[U-002 in the 3.0.5 guide](v3.0.5.md#u-002-grants-and-dependent-objects-do-not-survive-an-install).
87+
88+
## See also
89+
90+
- [`tests/timing/ore_block_256/`](../../tests/timing/ore_block_256/README.md):
91+
the timing harness behind U-001, and its results.

0 commit comments

Comments
 (0)