You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Oct 14, 2026. It is now read-only.
Repository navigation
Commit c784590
Browse filesBrowse the repository at this point in the historyBrowse files
refactor(data-access): remove copyEntityVectors and COPY_VECTORS_RPC (LLMO-7445) (#1964)
<!-- mysticat-pr-skill -->
## 1. Abstract
Removes the `copyEntityVectors` helper and the `COPY_VECTORS_RPC`
constant from data-access's semantic-index utils.
## 2. Reasoning
The helper's only caller was the offsite snapshot vector copy in
audit-worker, which is being removed (topic vectors are no longer copied
to the `IGNORED` superseded-run snapshot). With no caller left, the
helper and the `wrpc_copy_opportunity_semantic_vectors` RPC it wraps are
dropped, so the upcoming semantic-index redesign doesn't have to carry
them.
## 3. High-level overview of the changes
- `copyEntityVectors` and `COPY_VECTORS_RPC` are removed from
`semantic-index.utils.js` and from the package exports.
- The README example and the ADR-0002 description are updated.
- No other semantic-index helper changes behaviour.
- **Compatibility:** this removes a published export. Its only known
consumer, audit-worker, stops importing it in the PR linked below, so
this is not marked as a breaking change. A consumer still importing it
would fail at import time after upgrading.
## 4. Required information
- Jira / issue:
[LLMO-7445](https://jira.corp.adobe.com/browse/LLMO-7445)
- ADR(s): `docs/adr/0002-embedding-client-and-semantic-index-utils.md`
## 6. Affected / used mysticat-workspace projects
- spacecat-audit-worker: consumed. It must no longer import
`copyEntityVectors` before it upgrades to this release (done in its PR
below).
- mysticat-data-service: contract.
`wrpc_copy_opportunity_semantic_vectors` loses its last client wrapper;
dropped in the companion migration PR.
## 8. Test plan
(a) No runtime behaviour to verify beyond the removed helper. No local
e2e was run for this PR.
(b) After release: `grep -r copyEntityVectors` in the consuming repos
(audit-worker, api-service) should return nothing once audit-worker's PR
is merged.
## 9. Deployment & merge order
- Depends on:
https://github.com/Adobe-AEM-Sites/spacecat-audit-worker/pull/25
(removes the only import). Merge that first.
- Related: the mysticat-data-service PR dropping
`wrpc_copy_opportunity_semantic_vectors`.
- Order: audit-worker PR (merged and deployed), then this PR, then the
data-service migration.
🤖 Generated with
[review-kit](https://github.com/adobe/experience-success-skills)
Co-authored-by: Tanase Butcaru <tbutcaru@adobe.com>
Copy file name to clipboardExpand all lines: docs/adr/0002-embedding-client-and-semantic-index-utils.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -45,7 +45,7 @@ Following ADR-0001's test (*true for every consumer → shared; specific to one
45
45
46
46
### 4. Semantic-index helpers live in `data-access`, mirroring the URL index
47
47
48
-
`semantic-index.utils.js` mirrors `url-index.utils.js`: `syncOpportunitySemantic` (full-replace per `source_type`), `lookupOpportunitiesByVectors` (ANN read RPC wrapper, many query vectors per call), the batched `semantic_query_embedding` cache helpers (`getQueryEmbeddings` / `upsertQueryEmbeddings` / `touchQueryEmbeddings`), and `copyEntityVectors` (a thin wrapper over the `wrpc_copy_opportunity_semantic_vectors`**server-side** write RPC — the copy runs as `INSERT … SELECT`, no rows materialized in the caller). Text normalization + hashing live in this layer so a value written always matches a later read/re-sync. **The shared layer stores and reads pre-embedded vectors; it never embeds** — embedding is the writer's (audit-worker's) job via the client above.
48
+
`semantic-index.utils.js` mirrors `url-index.utils.js`: `syncOpportunitySemantic` (full-replace per `source_type`), `lookupOpportunitiesByVectors` (ANN read RPC wrapper, many query vectors per call), and the batched `semantic_query_embedding` cache helpers (`getQueryEmbeddings` / `upsertQueryEmbeddings` / `touchQueryEmbeddings`). (`copyEntityVectors`, a wrapper over the `wrpc_copy_opportunity_semantic_vectors` RPC, was removed with its only caller, the offsite snapshot vector copy.) Text normalization + hashing live in this layer so a value written always matches a later read/re-sync. **The shared layer stores and reads pre-embedded vectors; it never embeds** — embedding is the writer's (audit-worker's) job via the client above.
49
49
50
50
**Type registries.** `OPPORTUNITY_SEMANTIC_SOURCE_TYPES` (kinds of embedded text: `topic`, later `claim`) and `OPPORTUNITY_SEMANTIC_ENTITY_TYPES` (opportunity types, from `OPPORTUNITY_TYPES`) are defined in `spacecat-shared-utils` (so non-Node consumers such as the UI can import them and their types) and re-exported from this layer, and this library's writer stores and reader accepts only their values; anything else is a `ValidationError`. The reader takes `sourceTypes` (required, non-empty) and optional `entityTypes` (omitted or empty = all), deduped, so a list can never exceed its registry. The DB does not duplicate the registries: the RPC only rejects empty, multi-dimensional, or NULL-element lists and lists over 100 elements (a length bound, not a registry mirror; `NULL` entity types = all; the reader sends `NULL` when `entityTypes` is omitted or empty). A team that wants its opportunities searchable adds its source kind and/or opportunity type to the registries, indexes with them, and callers opt in by passing them.
0 commit comments