Skip to content

Commit 6b9b42c

Browse files
authored
test(golang): prove the EQL v4 types from Go, against Rust and PostgreSQL
stashgen needed no change: it maps int32, int64, bool, float32, float64 and string to their kinds and accepts any type the engine's se_targets table marks producible, so every type the previous commits made producible already generates typed Encrypt and Query methods. A Profile fixture stores one field in each new type. The hermetic test seals it through generated code over the deterministic EQL guest, checks every value is a v4 envelope with the type's keys, and opens it back. A cross-language fixture, v4_queries.json, holds the IntegerEq, BigintEq and TextMatch query operands the Rust dispatch derives under the shared test index key; the Go fields derive them byte for byte. Checked by hand against PostgreSQL: rows sealed by this Go code, loaded into an eql_v4 install, answer =, = and @@ with exactly the expected ids. The Go module has no PostgreSQL driver, and adding one for a test would change every consumer's dependency graph, so that check is not committed. The encrypt and stashgen READMEs name eql_v4 and list the producible types with the Go field type each takes. Refs #1144
1 parent ee41f6e commit 6b9b42c

7 files changed

Lines changed: 530 additions & 4 deletions

File tree

‎languages/golang/cmd/stashgen/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ Your program calls those functions, and it never builds or names a plan.
4444
```
4545

4646
6. Store the encrypted type.
47-
Each `encrypt_into` field is one EQL column: `eql.TextEq` implements `driver.Valuer` and `sql.Scanner`, so a database library binds and scans it as the JSON a `public.eql_v3_text_eq` column holds.
47+
Each `encrypt_into` field is one EQL column: `eql.TextEq` implements `driver.Valuer` and `sql.Scanner`, so a database library binds and scans it as the JSON a `public.eql_v4_text_eq` column holds. Stack Encrypt writes EQL v4; see the `encrypt` package README for the EQL types it produces and the Go field type each takes.
4848

4949
7. Run the generator again after each change to the struct or to a tag.
5050
A change to the fields of the struct stops the build until you do.

‎languages/golang/encrypt/README.md‎

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -273,12 +273,32 @@ package documentation for the full account.
273273
## EQL columns
274274

275275
A field tagged `encrypt_into=TextEq` is stored as one EQL value, the JSON
276-
PostgreSQL holds in a `public.eql_v3_text_eq` column, as an `eql.TextEq`
277-
from package `encrypt/eql`. The guest builds the value: the generated code
276+
PostgreSQL holds in a `public.eql_v4_text_eq` column, as an `eql.TextEq`
277+
from package `encrypt/eql`. Stack Encrypt writes EQL v4, the `eql_v4` install
278+
of EQL's SQL; the `eql_v3` columns the TypeScript SDK writes hold
279+
cipherstash-client's terms, which these never match.
280+
281+
The EQL types the engine produces, and the Go field types each takes:
282+
283+
| EQL type | Go field | Queries |
284+
|---|---|---|
285+
| `TextEq` | `string` | `=`, `<>` |
286+
| `TextMatch` | `string` | `@@` (substring match, 3-grams, case-insensitive) |
287+
| `Text` | `string` | none: stored and read only |
288+
| `IntegerEq` | `int32` | `=`, `<>` |
289+
| `Integer` | `int32` | none |
290+
| `BigintEq` | `int64` | `=`, `<>` |
291+
| `Boolean` | `bool` | none |
292+
| `Real` | `float32` | none |
293+
| `Double` | `float64` | none |
294+
295+
The rest of the catalog (ordering, dates, decimals, JSON) is listed in
296+
`eql.Types` with the reason it is not produced yet, and `stashgen` refuses it
297+
with that reason. The guest builds the value: the generated code
278298
imports `encrypt/eql`, which embeds the build of the engine that holds the
279299
EQL types and registers it on import, so the program runs that build and
280300
stores what it returns. `Fields.Email.Query` returns the `eql.TextEqQuery`
281-
the column's `eql_v3.query_text_eq` operand takes. `mise run
301+
the column's `eql_v4.query_text_eq` operand takes. `mise run
282302
wasm:guest:build:eql` builds that guest beside the other. A cipher extended
283303
with a tenant part refuses a struct with an `encrypt_into` field: an EQL
284304
value is stored under a table and a column, and the extended label has no
Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
package encrypt_test
2+
3+
import (
4+
"context"
5+
"encoding/json"
6+
"os"
7+
"path/filepath"
8+
"sort"
9+
"strings"
10+
"testing"
11+
12+
"github.com/cipherstash/stack/languages/golang/encrypt/internal/testusers"
13+
)
14+
15+
// The EQL v4 types the engine produces besides TextEq, sealed through
16+
// generated code over the deterministic guest build with the EQL types:
17+
// each Go field type reaches its v4 type's own plan, and the values open
18+
// back to what went in.
19+
20+
var profiles = []testusers.Profile{
21+
{ID: 1, Age: 42, Visits: 3, Balance: 1234567890123, Active: true, Ratio: 0.5, Score: 2.25, Note: "a private note", Name: "alice"},
22+
{ID: 2, Age: -7, Visits: 0, Balance: -1, Active: false, Ratio: -1.5, Score: 0, Note: "", Name: "bob smith"},
23+
}
24+
25+
// keysOf is the sorted top-level keys of an EQL value.
26+
func keysOf(t *testing.T, value []byte) string {
27+
t.Helper()
28+
var raw map[string]json.RawMessage
29+
if err := json.Unmarshal(value, &raw); err != nil {
30+
t.Fatalf("the EQL value is not JSON: %v: %s", err, value)
31+
}
32+
if string(raw["v"]) != "4" {
33+
t.Errorf("v = %s, want 4: %s", raw["v"], value)
34+
}
35+
keys := make([]string, 0, len(raw))
36+
for k := range raw {
37+
keys = append(keys, k)
38+
}
39+
sort.Strings(keys)
40+
return strings.Join(keys, ",")
41+
}
42+
43+
func TestV4FieldsSealToTheirEQLTypesAndOpenBack(t *testing.T) {
44+
c := deterministicEQLClient(t)
45+
ctx := context.Background()
46+
cipher := c.DefaultKeyset()
47+
48+
encrypted, err := testusers.EncryptProfile(ctx, cipher, profiles)
49+
if err != nil {
50+
t.Fatal(err)
51+
}
52+
for i, e := range encrypted {
53+
for name, tc := range map[string]struct {
54+
value []byte
55+
keys string
56+
}{
57+
"age": {e.Age, "c,hm,i,v"},
58+
"visits": {e.Visits, "c,i,v"},
59+
"balance": {e.Balance, "c,hm,i,v"},
60+
"active": {e.Active, "c,i,v"},
61+
"ratio": {e.Ratio, "c,i,v"},
62+
"score": {e.Score, "c,i,v"},
63+
"note": {e.Note, "c,i,v"},
64+
"name": {e.Name, "bf,c,i,v"},
65+
} {
66+
if got := keysOf(t, tc.value); got != tc.keys {
67+
t.Errorf("row %d %s: keys %s, want %s", i, name, got, tc.keys)
68+
}
69+
}
70+
}
71+
72+
decrypted, err := testusers.DecryptProfile(ctx, cipher, encrypted)
73+
if err != nil {
74+
t.Fatal(err)
75+
}
76+
for i := range profiles {
77+
if decrypted[i] != profiles[i] {
78+
t.Errorf("row %d: %+v, want %+v", i, decrypted[i], profiles[i])
79+
}
80+
}
81+
}
82+
83+
func TestV4QueriesMatchTheRustFixture(t *testing.T) {
84+
c := deterministicEQLClient(t)
85+
ctx := context.Background()
86+
cipher := c.DefaultKeyset()
87+
88+
raw, err := os.ReadFile(filepath.Join("..", "..", "..", "packages", "eql", "tests", "encryption", "fixtures", "v4_queries.json"))
89+
if err != nil {
90+
t.Fatal(err)
91+
}
92+
var fixture struct {
93+
Table string
94+
Age struct {
95+
Plaintext int32
96+
Query string
97+
}
98+
Balance struct {
99+
Plaintext int64
100+
Query string
101+
}
102+
Name struct {
103+
Plaintext string
104+
Query string
105+
}
106+
}
107+
if err := json.Unmarshal(raw, &fixture); err != nil {
108+
t.Fatal(err)
109+
}
110+
if fixture.Table != "profiles" {
111+
t.Fatalf("the fixture's table changed: %q", fixture.Table)
112+
}
113+
114+
// Byte for byte what eql-bindings' own dispatch derived under the same
115+
// index key: each Go field runs its v4 type's plan and no other.
116+
age, err := testusers.ProfileFields.Age.Query(ctx, cipher, fixture.Age.Plaintext)
117+
if err != nil {
118+
t.Fatal(err)
119+
}
120+
if string(age) != fixture.Age.Query {
121+
t.Errorf("Age.Query = %s\nwant %s", age, fixture.Age.Query)
122+
}
123+
balance, err := testusers.ProfileFields.Balance.Query(ctx, cipher, fixture.Balance.Plaintext)
124+
if err != nil {
125+
t.Fatal(err)
126+
}
127+
if string(balance) != fixture.Balance.Query {
128+
t.Errorf("Balance.Query = %s\nwant %s", balance, fixture.Balance.Query)
129+
}
130+
name, err := testusers.ProfileFields.Name.Query(ctx, cipher, fixture.Name.Plaintext)
131+
if err != nil {
132+
t.Fatal(err)
133+
}
134+
if string(name) != fixture.Name.Query {
135+
t.Errorf("Name.Query = %s\nwant %s", name, fixture.Name.Query)
136+
}
137+
138+
// And a stored value carries the term its query probes for.
139+
stored, err := testusers.ProfileFields.Age.Encrypt(ctx, cipher, fixture.Age.Plaintext)
140+
if err != nil {
141+
t.Fatal(err)
142+
}
143+
var s, q struct {
144+
HM string `json:"hm"`
145+
}
146+
if err := json.Unmarshal(stored, &s); err != nil {
147+
t.Fatal(err)
148+
}
149+
if err := json.Unmarshal(age, &q); err != nil {
150+
t.Fatal(err)
151+
}
152+
if s.HM != q.HM {
153+
t.Errorf("stored hm %s, query hm %s", s.HM, q.HM)
154+
}
155+
}

0 commit comments

Comments
 (0)