feat(search): weighted FTS ranking + searchable message bodies - #294
feat(search): weighted FTS ranking + searchable message bodies#294salmonumbrella wants to merge 1 commit into
Conversation
|
Thanks, this is great. The security side is clean too, I checked the FTS and SQL paths hard and there's no injection, and the auth and per-user scoping are right. A few things before I can merge: Blocking
Should fix
Everything else I found is minor and can come later. Once 1 through 3 are in I'll re-review. |
Results ranked by relevance instead of date-only: subject > sender > body (setweight A-D + ts_rank_cd), prefix matching as you type, quoted phrases, served by a GIN index over a trigger-maintained search_fts column. Backfill runs as a resumable background drainer (fast-DDL migrations 0035-0037, no boot-blocking rewrite); not-yet-backfilled rows fall back to the previous query path. Body text is materialized by a provider-gated background IMAP drainer with circuit breakers, poison-message forward progress, and progress reporting. Filter-only queries stay date-ordered. No infrastructure changes - runs on stock postgres:16-alpine. Split 1/3 of maathimself#283. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
42fc2d4 to
ef15a6d
Compare
|
fixed the four review points in
verification:
CI note: both GitHub jobs currently stop at the audit step because unchanged current- |
Split 1/3 of #283, per the review request there. Keyword-search improvements only — no infrastructure changes, no new dependencies, runs on stock
postgres:16-alpine.Summary
Search results are ranked by relevance instead of date-only, and message bodies become searchable across the whole mailbox.
setweightA–D +ts_rank_cd), prefix matching as you type, quoted phrases, served by a GIN index over a trigger-maintainedsearch_ftscolumn. Filter-only queries stay date-ordered.search_ftsfills via a background drainer, and not-yet-backfilled rows fall back to the previous query path. The legacysearch_vectorcolumn stays until a follow-up removes it after real-install validation.body_text(Apple/Yahoo/generic on; Gmail/PurelyMail/Microsoft off given their throttle history), with pacing, poison-message forward progress, and per-account circuit breakers.mode: 'lexical'field — the stable contract the semantic follow-up PR extends with real modes.Review fixes
main; migrations are now 0041–0043.include_in_unified_inbox.SET LOCAL statement_timeout = 0, so large mailboxes do not restart the same count every 30 seconds.Testing
Contributor License Agreement
By submitting this pull request I confirm that: