Skip to content

Add live duplicate check to the new-entry dialog#2411

Closed
myieye wants to merge 5 commits into
developfrom
feat/possible-duplicates
Closed

Add live duplicate check to the new-entry dialog#2411
myieye wants to merge 5 commits into
developfrom
feat/possible-duplicates

Conversation

@myieye

@myieye myieye commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

Searches existing entries as you type in the new-entry dialog and surfaces likely duplicates before you save.

Resolves #1752

  • duplicate-check.ts — debounced multi-field search (headword/citation/gloss) via the existing FTS searchEntries; results classified as same word / similar word / similar meaning and ranked strongest-first (exact match, then closest length, then gloss overlap). Cross-field coincidences (a typed vernacular value that only hit a gloss, or vice versa) are dropped as noise. Classification approximates the backend match semantics (invariant case fold; diacritics significant only when the typed text has them; morph tokens stripped like EntrySearchService.StripMorphTokens — CRDT FTS path only; FwData and the <3-char fallback don't strip); collation equivalences (ss/eszett, ligatures, ICU-ignorables) are not replicated — a backend folding change should re-check this file
  • DuplicateCheck.svelte — collapsible strip in NewEntryDialog.svelte (amber when the word already exists, muted otherwise); auto-expands on an exact word match; collapsed header previews the matched headwords; rows expand in place to reveal "Add sense" / "Go to entry" actions; "Show N more" for long lists; quiet green "Looks like a new word" line when nothing matches
  • DuplicateSummaryPill.svelte — sticky pill in the dialog once the strip scrolls out of view (IntersectionObserver), so a match found while typing lower fields isn't missed; click jumps back and expands the strip; dismissible
  • DictionaryEntry.svelte — new inline prop (senses flow on one line) for the compact collapsed rows
  • Per-row actions: jump to the existing entry, or add the typed sense to it instead of creating a duplicate. Add-sense saves only the meaning — other typed entry fields (e.g. citation form) are intentionally discarded, since the word already exists
  • Perf: each distinct typed value fires one debounced (300 ms) searchEntries fetching 20 rows (over-fetched because cross-field hits are filtered client-side) — typically 1-3 queries per pause; in-process on desktop, SignalR round-trips on server-hosted sessions (abort is client-side discard only; the signal can't cross JSInterop)
  • Keyboard: Enter inside the duplicate strip or pill activates the focused control instead of submitting the dialog; double-Enter can no longer double-create (pre-existing race, fixed in passing)
  • No backend changes

Note: the .po regen also picks up stale activity-view #: refs from an earlier refactor (benign, identical across locales).

Screenshots

image

Exact match:
image

Clicking scrolls to: (and auto-expands)
Note the "Same lexeme form" badge which explains why the displayed headword "gasa" does not exactly match the entered text "gas":
image

Rows are expandable, because:

  • It's less likely a user will accidentally lose progress by navigating to the potential duplicate
  • Provides a natural place to expand the whole dictionary preview
  • Provides a place for an "Add sense" button in case the headword matches AND sense data was entered
image

Not expanded automatically if only "similar" entries: (but clicking the sticky indicator always triggers an expand)

image

This:
image
expands to this:
image

(from an earlier iteration — rows now expand in place for their actions instead of navigating on click)
image
image

Test plan

  • 33 unit tests — query building, normalization parity (case/diacritics/morph tokens), classification order, field attribution, length-delta boundary; 8/8 targeted mutants killed against the current classifier
  • 7 Playwright tests — duplicates + row-expand navigation, new-word indicator, add-sense rescue, similar-word badge, Show-more/capped count, gloss badge, out-of-view jump pill
  • i18n extracted with translator context comments
  • Manually verified on desktop + mobile widths, light + dark

Considered and rejected

  • Wizard-style duplicate step — adds a click to the common no-duplicate case
  • Post-submit interstitial — warns only after the duplicate is typed in full; inline is cheaper to act on
  • Combobox under the headword field — ties the check to one field; duplicates can hide in citation form or gloss

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the 💻 FW Lite issues related to the fw lite application, not miniLcm or crdt related label Jul 4, 2026
@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: fb70578c-440c-4dde-8426-778c200e320d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a duplicate-entry detection feature to the entry editor: a new duplicate-check utility module classifies candidate entries by match kind, a DuplicateCheck.svelte component surfaces matches within NewEntryDialog.svelte with navigate/add-sense actions, plus unit tests, Playwright tests, and new localization strings across nine locale files.

Changes

Duplicate Check Feature

Layer / File(s) Summary
Duplicate matching utility and tests
frontend/viewer/src/lib/entry-editor/duplicate-check.ts, duplicate-check.test.ts
Introduces DuplicateMatchKind, DuplicateMatch, DuplicateQueries, normalizeForCompare, duplicateQueries, mergeSearchResults, classifyDuplicates, with unit tests covering normalization, query extraction, merging, and classification/ranking.
DuplicateCheck component and dialog wiring
frontend/viewer/src/lib/entry-editor/DuplicateCheck.svelte, NewEntryDialog.svelte
New component runs a debounced duplicate search, renders a collapsible match list with "Go to entry" and "Add sense/meaning" actions, and is embedded in NewEntryDialog.svelte, closing the dialog on navigation.
Playwright end-to-end tests
frontend/viewer/tests/new-entry-duplicates.test.ts
Adds tests for exact match, new word, add-sense persistence, partial headword match, capped list expansion, and gloss-only matching.
Localization strings
frontend/viewer/src/locales/{en,es,fr,id,ko,ms,sw,vi}.po
Adds duplicate-check msgids (status, badges, actions, success notifications, warnings) and updates source-reference comments for existing entries across all locale catalogs.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers: hahn-kev

Poem

A rabbit hops through fields of text,
Sniffing for words that look duplexed 🐇
"Same headword here, similar there!"
One click to add a sense with care.
No more twin entries lost in the maze—
Just tidy dictionaries, cause for praise!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: adding live duplicate checking to the new-entry dialog.
Description check ✅ Passed The description is directly related to the changeset and explains the duplicate-check flow in detail.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/possible-duplicates

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@argos-ci

argos-ci Bot commented Jul 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Argos notifications ↗︎

Build Status Details Updated (UTC)
default (Inspect) ✅ No changes detected - Jul 7, 2026, 7:33 PM

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
frontend/viewer/tests/new-entry-duplicates.test.ts (3)

21-27: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Selectors depend on internal inline style attributes.

lexemeInput/glossInput locate fields via [style*="grid-area: lexemeForm"] and [style*="grid-area: gloss"]. These are implementation details of the component's CSS grid layout, not stable testing contracts — any layout refactor (e.g. switching to a class-based grid or renaming grid areas) will silently break these locators.

♻️ Suggested approach

Add data-testid (or rely on accessible labels) on the lexeme/gloss form fields in the dialog component, then select via dialog.getByTestId('lexeme-form-field') etc. This decouples tests from layout implementation.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/viewer/tests/new-entry-duplicates.test.ts` around lines 21 - 27, The
current test helpers lexemeInput and glossInput are coupled to internal inline
style selectors, which makes the Playwright locators brittle. Update the dialog
component to expose stable selectors such as data-testid or accessible labels
for the lexeme and gloss fields, then change these helpers to use those stable
hooks instead of querying [style*="grid-area: ..."] so the new-entry-duplicates
test no longer depends on CSS layout details.

103-119: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Test hardcodes fixture-specific counts ("3", "10+") tightly coupled to demo dataset.

This test's assertions (toHaveCount(3), checking for the literal text '10+') depend on the exact number of demo entries whose headword starts with 'ba'. Any change to demo-entry-data.ts (adding/removing entries starting with "ba") will silently break this test without an obvious link to the root cause.

Consider deriving the expected counts from the demo dataset itself (e.g., counting matching entries programmatically) or adding an inline comment enumerating which demo entries are expected to match, to make the coupling more discoverable when it breaks.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/viewer/tests/new-entry-duplicates.test.ts` around lines 103 - 119,
The duplicate-list test is hardcoded to fixture-specific counts, so update the
assertions in new-entry-duplicates.test.ts to avoid depending on a fixed "3" or
literal "10+" tied to the demo dataset. Use the existing test helpers around
openNewEntryDialog, lexemeInput, duplicateRows, and duplicatesSummary to derive
the expected number of matches programmatically from the demo entries or
document the exact expected "ba" matches inline so changes in demo-entry-data.ts
are easier to track.

82-82: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Non-null assertion on entryId could produce a confusing failure if absent.

new URL(page.url()).searchParams.get('entryId')! would silently coerce null to satisfy the type checker if the param is missing, leading to an unclear downstream assertion failure via entryHasGlossValue rather than a clear "entryId missing" error.

🧪 Suggested tweak
-    const entryId = new URL(page.url()).searchParams.get('entryId')!;
+    const entryId = new URL(page.url()).searchParams.get('entryId');
+    expect(entryId).toBeTruthy();
     await expect(async () => {
-      expect(await browsePage.api.entryHasGlossValue(entryId, newGloss)).toBe(true);
+      expect(await browsePage.api.entryHasGlossValue(entryId!, newGloss)).toBe(true);
     }).toPass({timeout: 5000});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/viewer/tests/new-entry-duplicates.test.ts` at line 82, The `entryId`
lookup in `new-entry-duplicates.test.ts` is using a non-null assertion that can
hide a missing query param and cause a misleading failure later. Update the test
around `page.url()` and `searchParams.get('entryId')` to explicitly check for a
missing value and fail immediately with a clear message before calling
`entryHasGlossValue`, so the failure points directly to the absent `entryId`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@frontend/viewer/src/lib/entry-editor/DuplicateCheck.svelte`:
- Around line 113-128: The pending save in addSenseToEntry can overwrite a newer
user navigation because openEntry(target) always runs after createSense
resolves. Update the DuplicateCheck.svelte flow so a “go to entry” click during
an in-flight add-sense either waits/blocks, or the post-save navigation is
skipped if the user has already navigated elsewhere; use the existing
addingSense state and the entry-opening logic around addSenseToEntry/openEntry
to keep the final navigation aligned with the user’s last click.

---

Nitpick comments:
In `@frontend/viewer/tests/new-entry-duplicates.test.ts`:
- Around line 21-27: The current test helpers lexemeInput and glossInput are
coupled to internal inline style selectors, which makes the Playwright locators
brittle. Update the dialog component to expose stable selectors such as
data-testid or accessible labels for the lexeme and gloss fields, then change
these helpers to use those stable hooks instead of querying [style*="grid-area:
..."] so the new-entry-duplicates test no longer depends on CSS layout details.
- Around line 103-119: The duplicate-list test is hardcoded to fixture-specific
counts, so update the assertions in new-entry-duplicates.test.ts to avoid
depending on a fixed "3" or literal "10+" tied to the demo dataset. Use the
existing test helpers around openNewEntryDialog, lexemeInput, duplicateRows, and
duplicatesSummary to derive the expected number of matches programmatically from
the demo entries or document the exact expected "ba" matches inline so changes
in demo-entry-data.ts are easier to track.
- Line 82: The `entryId` lookup in `new-entry-duplicates.test.ts` is using a
non-null assertion that can hide a missing query param and cause a misleading
failure later. Update the test around `page.url()` and
`searchParams.get('entryId')` to explicitly check for a missing value and fail
immediately with a clear message before calling `entryHasGlossValue`, so the
failure points directly to the absent `entryId`.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 74e64c61-0469-474a-9d9b-58ef682e3de4

📥 Commits

Reviewing files that changed from the base of the PR and between a9cf4ca and f794f8f.

📒 Files selected for processing (13)
  • frontend/viewer/src/lib/entry-editor/DuplicateCheck.svelte
  • frontend/viewer/src/lib/entry-editor/NewEntryDialog.svelte
  • frontend/viewer/src/lib/entry-editor/duplicate-check.test.ts
  • frontend/viewer/src/lib/entry-editor/duplicate-check.ts
  • frontend/viewer/src/locales/en.po
  • frontend/viewer/src/locales/es.po
  • frontend/viewer/src/locales/fr.po
  • frontend/viewer/src/locales/id.po
  • frontend/viewer/src/locales/ko.po
  • frontend/viewer/src/locales/ms.po
  • frontend/viewer/src/locales/sw.po
  • frontend/viewer/src/locales/vi.po
  • frontend/viewer/tests/new-entry-duplicates.test.ts

Comment on lines +113 to +128
async function addSenseToEntry(target: IEntry): Promise<void> {
if (!sense || addingSense) return;
addingSense = true;
try {
const senseSnapshot = {...$state.snapshot(sense), entryId: target.id};
await saveHandler.handleSave(() => lexboxApi.createSense(target.id, senseSnapshot));
} finally {
addingSense = false;
}
AppNotification.display(
pt($t`Sense added to "${writingSystemService.headword(target)}"`,
$t`Meaning added to "${writingSystemService.headword(target)}"`,
viewService.currentView),
{type: 'success', timeout: 'short'});
openEntry(target);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Race: clicking "go to entry" on another row while an add-sense save is pending can trigger a surprise second navigation.

addingSense only disables the add-sense icon buttons; the "go to entry" buttons stay clickable. If a user clicks a different match's "go to entry" while addSenseToEntry is still awaiting createSense, the dialog closes and navigation happens immediately — but once the pending save resolves, addSenseToEntry still calls openEntry(target) again, silently re-navigating the user to a different entry than the one they just chose.

🛠️ Suggested fix
               <button
                 type="button"
                 class="grow min-w-0 flex items-center gap-2 rounded bg-background/80 hover:bg-accent px-2.5 py-2 text-start"
                 title={goToLabel}
                 aria-label={headword ? `${goToLabel}: ${headword}` : goToLabel}
+                disabled={addingSense}
                 onkeydown={trapEnter}
                 onclick={() => openEntry(match.entry)}>

Also applies to: 176-193

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@frontend/viewer/src/lib/entry-editor/DuplicateCheck.svelte` around lines 113
- 128, The pending save in addSenseToEntry can overwrite a newer user navigation
because openEntry(target) always runs after createSense resolves. Update the
DuplicateCheck.svelte flow so a “go to entry” click during an in-flight
add-sense either waits/blocks, or the post-save navigation is skipped if the
user has already navigated elsewhere; use the existing addingSense state and the
entry-opening logic around addSenseToEntry/openEntry to keep the final
navigation aligned with the user’s last click.

@hahn-kev

hahn-kev commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

I like it!

myieye and others added 2 commits July 6, 2026 09:08
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Spec-cited review against EntrySearchService/SqlHelpers found the client
fold diverged: host-locale lowercasing, unconditional accent-stripping
(backend keeps diacritics significant when the query has them), and no
morph-token handling (typed "-aji" vs suffix entry "aji" missed exact).
Also ranks each search by the writing system it was typed in, fixes the
mount-time "Checking" flash, singularizes the one-match banner, and
closes a pre-existing double-Enter double-create in the dialog.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@myieye myieye marked this pull request as draft July 6, 2026 08:26
myieye and others added 2 commits July 7, 2026 20:23
- match rows expand to reveal Add sense / Go to entry; DictionaryEntry gains an inline mode for the compact collapsed rows
- classification: attribute same-word matches to the field that hit (Same headword vs Same lexeme form), rank similar words closest-in-length first, drop cross-field coincidences instead of a vague 'related' kind
- classify in a $derived so the lazy morph-types resource warms at mount and re-classifies without re-searching
- dedupe queries per field kind: the same text typed as lexeme and gloss keeps its gloss query (same-meaning matches were lost)
- share the banner message/tint/Enter-trap between the strip and the extracted DuplicateSummaryPill
- scope backend-parity doc claims to the CRDT FTS path; drop unverified FLEx attribution
- adapt the Playwright suite to the new interaction model and cover the jump pill (7 tests); pin length-delta boundary, gloss containment direction, and closest-form ranking in unit tests (33; 8/8 targeted mutants killed)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…icates

# Conflicts:
#	frontend/viewer/src/locales/en.po
#	frontend/viewer/src/locales/es.po
#	frontend/viewer/src/locales/fr.po
#	frontend/viewer/src/locales/id.po
#	frontend/viewer/src/locales/ko.po
#	frontend/viewer/src/locales/ms.po
#	frontend/viewer/src/locales/sw.po
#	frontend/viewer/src/locales/vi.po
@myieye

myieye commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #2433 — same feature rebased onto latest develop (the #1866 test restructure had orphaned this branch's Playwright suite) plus review fixes: pill max-width, silent-search-failure surfacing, $plural, and extra test coverage. The race flagged by CodeRabbit below was already fixed here in ad8bfa5 and is re-verified there.


Generated by Claude Code

@myieye myieye closed this Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

💻 FW Lite issues related to the fw lite application, not miniLcm or crdt related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show Existing Entries in Create Entry Dialog to Prevent Duplicate

2 participants