Skip to content

Add offline local-current brain search to the CLI - #656

Closed
futurepaul wants to merge 1 commit into
masterfrom
brain-local-search
Closed

futurepaul wants to merge 1 commit into
masterfrom
brain-local-search

Conversation

@futurepaul

Copy link
Copy Markdown
Collaborator

Part of #627.

Adds fragment search <name> '<words>' --dir <folder> [--limit N], with the CLI's normal --json envelope. It reads current working Markdown offline, with no login, brain server, sync pass or watcher. Unsynced edits and local-only files are immediately searchable; deleted files disappear between invocations.

The disposable index is SQLite FTS5 in memory, rebuilt on every call. All query words are required prefixes; weighted BM25 ranks title (4), heading (3), path/ancestry (2) and body (1). Results identify fragment, path, title, heading, ancestry, excerpt and synced, local_only or conflicted state. Sync records divergence in an optional journal field so an unsuccessful remote-copy write cannot make adopted local bytes appear agreed. Existing conflict copies and merge markers also warn; remote copies are not separate truth-bearing hits.

Validation:

  • cargo test -p fragment-cli --offline: CLI unit tests and real-binary integration tests, including offline same-size edits, local additions, deletion/restart, journal and sibling-copy conflicts, merge markers, ranking/prefixes, limits, skipped files, invalid input and symlink exclusion.
  • cargo xtask check: JavaScript syntax/guards, schema lock, workspace tests, host and Wasm Clippy with warnings denied.
  • git diff --check.

Caveats: synced means matching the last local journal, not a fresh remote/published-content check. Each call reads all eligible pages; concurrent edits are not an atomic filesystem snapshot. Sync's ignore rules apply. Pages >=1 MiB, non-UTF-8 content and sections beyond the first 500 are reported as skipped. The CommonMark parser includes Setext headings, scalar frontmatter and inline tags; multiline YAML tags are not interpreted, and long sections split at 8000 Unicode characters. Older journals cannot identify a conflict whose remote copy was never saved. Full command/output and conflict semantics are documented in the CLI guide and brain README.

Scoped to local lexical search: no embeddings, services, path ACLs, collections, cell API changes or deployment.

@futurepaul

Copy link
Copy Markdown
Collaborator Author

The rerun of the failed jobs is green. All CI checks now pass: the unrelated chat-layout wait in shard 3 cleared, and brain-search checks plus all four e2e shards pass.

@futurepaul

Copy link
Copy Markdown
Collaborator Author

Closing per Paul (2026-10-30): search goes through the platform. A fragment's own index (#783, on the LanceDB service #777) gets the fragment search name, and agents already search well on their own. A local search may come back later as a separate, differently named command. The branch is kept.

@futurepaul futurepaul closed this Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant