Token API: server-side kNN /similarity endpoint + concurrency guard (don't starve live users)

[JasonWildMe]

The headline of the API-feedback series (MMF/Sharkbook): a server-side k-NN re-ID endpoint that kills the critical scaling bottleneck (no more downloading every 2,152-float vector — a 200-record page took ~176s), plus a concurrency guard so a high-volume agent can do unlimited work without crowding out interactive Wildbook users.

POST /api/v3/similarity (token, read-only)

Given an annotation the caller can see, returns the most visually similar annotations with scores and no vectors — surfacing Wildbook's existing OpenSearch kNN.

• Filtered ANN: ACL + same viewpoint + method/methodVersion + optional taxonomy/locationId scope + self/encounter exclusion all go inside the native knn.filter (an outer bool.filter would post-filter ANN results → recall loss/leak). Non-admins additionally get an outer ACL wrap (defense in depth). Filtered nested kNN was verified empirically on the live cluster (knn.filter honors nested and parent fields).
• Never returns vectors (_source excludes embeddings + defensive remove()).
• No-oracle visibility gate on the query annotation (not-visible == not-found == 404).
• Deterministic single-embedding resolution; viewpoint required; k 1–100.
• Re-ID metric wildbook_token_reid_queries_total — this is user-run identification compute on our servers, kept distinct from the pipeline wildbook_identification_tasks* gauges.

Concurrency guard (TokenApiConcurrency) — "unlimited volume, no interference"

Per-instance semaphores cap how many token requests run at once (not total volume); excess gets 429 + Retry-After. Wired into all token endpoints: a tight SIMILARITY pool (default 2) for the heavy kNN, a GENERAL pool (default 8) for search + media/resolve. Only token requests are throttled — interactive session/UI traffic is never. AutoCloseable permits (no leak/over-release, released on every path incl. exceptions), best-effort inflight/rejected Prometheus gauges. Sizes via OpenSearch.properties (restart to resize).

Request-scoped PITs (OpenSearch.queryPitPrivate)

The similarity endpoint uses a private point-in-time and never touches the shared static PIT_CACHE, so a token call can't delete or race a PIT a live UI search is using.

Process / tests

Both specs Codex design-reviewed before coding; code Codex-reviewed (3 rounds, converged). 46 tests across the touched suites: TokenApiConcurrencyTest (3), SimilarityApiTest (9: auth/validation, embeddings-stripped, ambiguous-embedding 400, filtered-ANN shape + self-exclusion, no-oracle 404, 429), plus GENERAL-guard 429 tests on MediaResolveApi/SearchApi.

Merge notes / fast-follows

• Touches SearchApi.java and MediaResolveApi.java (guard wiring) and OpenSearch.java; expect small, easily-resolved conflicts with media/resolve: explicit per-ID status (identified/unidentified/no_image/unavailable/error) #1646/Token search: safe terms aggregations (+ explicit 400 for unsupported aggs) #1647 — resolve in favor of keeping both changes.
• Recommended fast-follow (somewhat urgent): migrate the existing token search/media/resolve paths to queryPitPrivate too — today they delete shared PIT_CACHE entries and can disrupt live UI searches even at low concurrency. This PR fixes that for the new endpoint and bounds concurrency everywhere; the PIT migration for the older paths is the remaining piece.
• Platform follow-up (out of scope): a dedicated OpenSearch read pool/replica + edge rate-limit for stronger isolation.

🤖 Generated with Claude Code