Skip to content

Fix live search tests to treat result count as a limit - #1214

Merged
ebhills merged 1 commit into
mainfrom
codex/search-result-count-tests
Sep 30, 2026
Merged

ebhills merged 1 commit into
mainfrom
codex/search-result-count-tests

Conversation

@ebhills

@ebhills ebhills commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Linked issue

Closes #1213. Follow-up to the unrelated search-test failure after #1212.

What changes

The live search test requested seven results and received six, causing an incorrect exact-count failure. Live assertions now accept successful, nonempty results up to the requested limit and still reject provider errors. The multiple-query test checks every query response.

Adds offline cases for provider counts of 0, 6, 7, and 9 with a limit of 7, verifying returned counts of 0, 6, 7, and 7, result ordering, row identity, and the outgoing limit. Runtime behavior is unchanged.

How it was verified

  • Focused offline search suite: 57 passed.
  • Credential-free local suite (scripts/test-local.ps1): 2,680 passed, 6 skipped, 139 deselected.
  • Executed all 13 original TestFindLinks methods against a synthetic six-result provider with network access blocked; all passed. This isolated check loaded only the search test class, excluding live retrieval setup.
  • git diff --check passed. Two-file scope reviewed; branch current with freshly fetched origin/main.
  • Original live CI failure: test_search_with_n_results, six results versus an exact expectation of seven.
  • Fresh credentialed CI is pending; no live provider or deployment validation was performed locally.

Compatibility and risk

Test-only change in two existing files; no public API, runtime, schema, configuration, credential, or deployment changes. The live tests continue requiring successful, nonempty responses for their normal queries. Offline tests cover a valid empty provider response independently. Revert this commit to roll back.

Ready-for-review checklist

  • One human delivery owner is assigned
  • The linked issue and intended milestone are correct (no release milestone assigned)
  • The branch is current with main and has no merge conflicts
  • Focused tests pass
  • New or changed behavior has direct test coverage
  • Documentation/schema/configuration is updated where applicable (not applicable)
  • The PR contains no unrelated changes
  • The PR description reflects the branch's current scope and latest validation
  • One primary reviewer is requested only when this PR is ready

See the pull request workflow.

@ebhills ebhills self-assigned this Sep 30, 2026
@ebhills
ebhills marked this pull request as ready for review September 30, 2026 19:10
@ebhills
ebhills merged commit c5b2837 into main Sep 30, 2026
18 checks passed
@mborodii-prog
mborodii-prog deleted the codex/search-result-count-tests branch October 5, 2026 10:17
@ebhills ebhills mentioned this pull request Oct 11, 2026
16 of 40 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix live search tests that assume an exact result count

1 participant