Skip to content

Merge pull request #592 from otomatty/main#650

Merged
otomatty merged 1 commit into
fix/lint-engine-rulesfrom
claude/address-pr-646-comments-VqCIx
Apr 19, 2026
Merged

Merge pull request #592 from otomatty/main#650
otomatty merged 1 commit into
fix/lint-engine-rulesfrom
claude/address-pr-646-comments-VqCIx

Conversation

@otomatty
Copy link
Copy Markdown
Owner

@otomatty otomatty commented Apr 19, 2026

chore: sync main to develop


Open in Devin Review

Summary by CodeRabbit

Release Notes

  • New Features

    • Added Wiki Schema feature to define wiki constitution/guidelines.
    • Lint engine detects wiki quality issues with resolution workflows.
    • Activity log tracks user actions and wiki updates.
    • Auto-generates wiki index pages for content organization.
    • YouTube video direct clipping and metadata extraction.
    • "Promote to Wiki" dialog converts AI chat conversations to wiki pages.
    • Mobile bottom navigation for easier access on small screens.
  • Improvements

    • Enhanced rate limiting with per-endpoint policies.
    • Improved image upload cancellation support.
    • Better source and citation tracking for wiki content.

…, exact-dup severity

PR #646 のレビュー対応。

- conflict.ts: `NUMBER_PATTERN` に `(?: \d{3})*` を追加し、空白区切りの
  thousand grouping (`1 000 円` / `1 000 000 円`) を取りこぼさない。
  3 桁ずつのみ許容することで `100 3人` のような偽マッチは回避する。
  (coderabbitai #3107605897)

- conflict.ts: `normalizeNumberValue` を単位分離 + `parseFloat` ベースに
  書き換え、末尾の小数ゼロ (`1000` vs `1000.0`, `3.776` vs `3.7760`) を
  canonical 形式に畳む。
  (gemini-code-assist #3107607604)

- titleSimilar.ts: 完全一致 (`distance=0`) は重複ページ由来のリンク曖昧
  さを生むため `severity: "warn"` に上げ、近似一致 (`distance>0`) は
  従来どおり `info` のまま。finding 内の suggestion 文言も分岐させる。
  (gemini-code-assist #3107607603)

- conflict.test.ts: 空白区切り、百万単位、末尾 `.0`、小数末尾ゼロを
  同値として正規化するテストを追加 (vitest 29/29 passing)。

https://claude.ai/code/session_012J9gqETyditCqdU6xA6z87
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 19, 2026

Caution

Review failed

Failed to post review comments

📝 Walkthrough

Walkthrough

This PR introduces a comprehensive wiki intelligence and management system with five major components: an LLM-driven ingest planner for article integration, a multi-rule lint engine for wiki quality checks, an append-only activity log for audit trails, auto-generated index pages for wiki structure, and user-defined schema pages. It also adds client-side YouTube extraction with AI summaries, OCR and image description capabilities, MCP token revocation and rate limiting, a "promote to wiki" feature converting chat conversations to pages, and substantial layout refactoring (sidebar removal, mobile bottom navigation, responsive header).

Changes

Cohort / File(s) Summary
Configuration & Environment
.env.example, server/mcp/.env.example, package.json, server/api/package.json
Added environment variables for YouTube API key, MCP redirect URI allowlist, JWT expiration, and MCP server configuration. Added dependencies for tesseract.js, youtube-transcript, and react-email components.
Database Migrations
server/api/drizzle/000[7-11]_*.sql, server/api/drizzle/meta/_journal.json
Added five migrations: sources and page_sources tables for ingest data, lint_findings for quality checks, page is_schema flag for special pages, activity_log for audit trail, and page special_kind for index/log pseudo-pages.
Lint Engine
server/api/src/services/lintEngine/*
Implemented comprehensive lint rule system with orphan, ghost-link, title-similarity, broken-link, conflict-detection, and stale-content rules. Includes rule execution orchestration and finding persistence.
Ingest System
server/api/src/services/ingestPlanner.ts, server/api/src/lib/articleExtractor.ts, server/api/src/lib/youtubeExtractor.ts, server/api/src/lib/youtubeEmbedServerExtension.ts, server/api/src/services/youtubeService.ts
Added LLM-driven ingest planning pipeline with URL-to-content extraction, YouTube metadata/transcript fetching, AI-based entity extraction, and Tiptap-compatible JSON generation with YouTube embed support.
Activity & Index Services
server/api/src/services/activityLogService.ts, server/api/src/services/indexBuilder.ts
Implemented activity log recording with multiple action kinds and an auto-generated wiki index page builder with category sorting and markdown generation.
New API Routes
server/api/src/routes/ingest.ts, server/api/src/routes/lint.ts, server/api/src/routes/activity.ts, server/api/src/routes/wikiSchema.ts, server/api/src/routes/clip.ts, server/api/src/routes/mcp.ts
Added endpoints for ingest planning and application, lint execution and findings, activity log querying, wiki schema CRUD, YouTube clipping, and MCP token revocation with rate limiting.
Schema & Relations
server/api/src/schema/sources.ts, server/api/src/schema/pageSources.ts, server/api/src/schema/lintFindings.ts, server/api/src/schema/activityLog.ts, server/api/src/schema/pages.ts, server/api/src/schema/relations.ts, server/api/src/schema/index.ts
Extended database schema with sources, page_sources, lint_findings, and activity_log tables; added is_schema and special_kind columns to pages; defined all necessary Drizzle ORM relations.
MCP Auth & Rate Limiting
server/api/src/lib/mcpAuth.ts, server/api/src/middleware/mcpAuth.ts, server/api/src/middleware/rateLimit.ts
Implemented MCP token revocation with Redis deny-list, JWT expiration configuration, and restructured rate limiting with configurable per-user/per-IP limits and atomic Redis operations.
Admin UI Routes & Pages
admin/src/App.tsx, admin/src/pages/Layout.tsx, admin/src/pages/ActivityLog.tsx, admin/src/pages/wiki-health/*
Added new admin routes for wiki health dashboard and activity log. Wiki health displays lint findings with resolution actions; activity log shows filtered audit trail with kind/actor filtering.
Admin API Client
admin/src/api/activity.ts, admin/src/api/lint.ts
Added admin API methods for activity log querying, lint execution/findings retrieval, and finding resolution with proper error handling and type definitions.
Frontend Layout Restructuring
src/App.tsx, src/components/layout/AppLayout.tsx, src/components/layout/AppSidebar.tsx (deleted), src/components/layout/AppSidebarAiChatSection.tsx (deleted), src/components/layout/Header/index.tsx, src/components/layout/Header/MonthNavigation.tsx (deleted), src/components/layout/BottomNav/*, src/components/layout/MobileHeader.tsx, src/components/layout/NavigationMenu.tsx, src/components/layout/Header/UnifiedMenu.tsx
Removed left sidebar and month filtering, added mobile bottom navigation with Home/Notes/AI/Me tabs, restructured header with navigation menu and unified account menu, introduced responsive layout with CSS variables for safe area and nav heights.
Chat to Wiki Promotion
src/components/ai-chat/PromoteToWikiDialog.tsx, src/components/ai-chat/EntityRow.tsx, src/hooks/usePromoteToWiki.ts, src/lib/aiChat/extractEntitiesPrompt.ts, src/lib/aiChat/extractEntitiesPrompt.test.ts
Implemented UI flow to convert chat conversations to wiki pages via LLM entity extraction, with dialog UI for entity selection and page creation confirmation.
AI Chat UI Updates
src/components/ai-chat/AIChatMessage.tsx, src/components/ai-chat/AIChatMessages.tsx, src/components/ai-chat/AIChatPanelContent.tsx, src/components/ai-chat/AIChatHeader.tsx, src/components/ai-chat/AIChatViewTabs.tsx, src/components/ai-chat/AIChatConversationListRow.tsx (modified)
Added promote-to-wiki callback, desktop-only workflow tab gating, claude-code mode normalization on web, and simplified conversation list row to single page-style variant.
Image Processing
src/lib/ocr/tesseractOcr.ts, src/lib/ocr/tesseractOcr.test.ts, src/lib/ai/describeImage.ts, src/lib/ai/describeImage.test.ts, src/components/layout/ImageCreateDialog.tsx
Added client-side Tesseract OCR with language detection, AI-powered image description via multiple provider SDKs, and integrated both into image creation dialog with progress tracking and error handling.
Web Clipper Enhancements
src/hooks/useWebClipper.ts, src/lib/api/apiClient.ts, server/api/src/lib/clipAndCreate.ts
Extended web clipper to support YouTube URLs with optional AI summarization using persisted user AI settings, delegating to dedicated YouTube extraction pipeline.
Editor Extensions
src/components/editor/extensions/WikiLinkExtension.ts, src/components/editor/extensions/transformWikiLinksInContent.ts, src/components/editor/extensions/MarkdownPasteExtension.test.ts, src/components/editor/TiptapEditor/slashCommandItems.ts
Updated wiki-link handling to preserve bracket syntax in display text, updated slash command items with platform gating for executable code blocks and MCP resources (desktop only).
Storage & State
src/lib/storage/providers/*.ts, src/lib/storage/types.ts, src/lib/storageAdapter/IndexedDBStorageAdapter.ts, src/lib/onboardingState.test.ts
Added AbortSignal support to all storage providers for cancellable uploads, new ResetDatabasePageIdsReadError for graceful failure handling, onboarding state tests.
Utility Functions
src/lib/contentUtils.ts, src/lib/markdownExport.ts, src/lib/dateUtils.ts, src/lib/aiCostUtils.test.ts, src/lib/api/apiClient.test.ts, server/api/scripts/inspect-ai-models-cost.lib.ts
Added youtubeEmbed node type support, markdown export for YouTube embeds, date labeling and grouping utilities, cost multiplier utilities with tests, and cost baseline calculation.
Wiki Schema
src/hooks/useWikiSchema.ts, src/components/page/WikiSchemaPage.tsx (inferred), server/api/src/routes/wikiSchema.ts
Implemented per-user wiki schema (constitution) with React Query integration, GET/PUT endpoints, and activity logging.
Lint Suggestions
src/components/page/LintSuggestions.tsx
Added inline lint suggestions component for page editors, fetching and displaying findings with per-finding resolution actions.
Localization
src/i18n/*, src/i18n/locales/*/wikiSchema.json, src/i18n/locales/*/aiChat.json, src/i18n/locales/*/generalSettings.json, src/i18n/locales/*/nav.json, src/i18n/locales/*/home.json
Added wikiSchema translation domain for both EN/JA, added promote-to-wiki notifications, updated navigation strings (removed sidebar-specific entries, added subscription/account), removed month-filtering translations.
Documentation
AGENTS.md, README.md, server/mcp/README.md
Updated AGENTS.md with MCP server documentation, added server/mcp/README.md with deployment/configuration guide, updated README tech stack and project structure sections.
CI/CD & Workflows
.github/workflows/deploy-dev.yml, .github/workflows/deploy-prod.yml
Added verify-server-health job to both workflows for API and MCP endpoint health checking with retry logic.
MCP Server Updates
server/mcp/src/client/*.ts, server/mcp/src/tools/index.ts, server/mcp/src/__tests__/*
Added listPages MCP tool with pagination/scope support, rate-limit error handling with retry-after semantics, HTTP client pagination wiring, and comprehensive E2E/unit tests.
Comprehensive Test Coverage
server/api/src/__tests__/*, admin/src/api/*.test.ts, src/components/*/*.test.tsx, src/hooks/*.test.ts, src/lib/*/*.test.ts
Added extensive test suites for new services (activityLog, indexBuilder, ingestPlanner, lintRules), routes, API clients, UI components (PromoteToWikiDialog, BottomNav, MobileHeader), hooks, and utilities with mocking and integration test patterns.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

Suggested labels

enhancement, database, admin-ui, mobile, ai-integration, testing


🐰 A wiki learns, grows, and thinks,
With lint rules sharp and ingest links,
From chat to pages, OCR sees,
While bottom nav hops through the trees!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/address-pr-646-comments-VqCIx

@otomatty otomatty changed the base branch from main to fix/lint-engine-rules April 19, 2026 23:17
@chatgpt-codex-connector
Copy link
Copy Markdown

💡 Codex Review

const modelInfo = await validateModelAccess(body.model, tier, db);

P1 Badge Normalize model ID before usage accounting

This handler trims body.model for access validation, but later uses the raw body.model for validateModelAccess/recordUsage inside a non-fatal try/catch. If a client sends a valid model with surrounding whitespace (for example, " openai:gpt-4o "), the AI summary still runs, but usage recording fails and is swallowed, letting requests bypass monthly budget accounting for this endpoint.


const [newPage] = await tx
.insert(pages)
.values({

P2 Badge Resolve schema-page insert races with conflict handling

When two first-time PUT /api/wiki-schema requests run concurrently for the same user, both transactions can observe no existing schema row and attempt insert(pages); one then hits the per-owner schema unique index and fails with a 500. The current SELECT ... FOR UPDATE does not lock anything when no row exists, so this path needs ON CONFLICT (or retry/reselect) to avoid intermittent failures under concurrent saves.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

Open in Devin Review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements the 'LLM Wiki' pattern, adding immutable sources, an ingest pipeline with YouTube support, a wiki schema system, and a lint engine. It also refactors the layout into a consolidated navigation menu and mobile bottom nav, introduces client-side OCR and AI image description, and adds rate limiting for MCP endpoints. Feedback identifies bugs where ydocState is incorrectly initialized as empty for special pages and suggests reordering image uploads to prevent orphan S3 objects. Improvements are recommended for handling database conflicts, optimizing the title similarity lint rule, and restoring a missing AI history link in the navigation menu.

I am having trouble creating individual review comments. Click here to see my feedback.

server/api/src/services/indexBuilder.ts (329-334)

high

ydocState に空の Buffer をセットしているため、生成された __index__ ページをエディタで開くと内容が空に見えてしまいます。document.markdown を Tiptap JSON に変換し、prosemirrorJSONToYDoc を使用して有効な ydocState を生成して保存するように修正してください。

server/api/src/routes/wikiSchema.ts (128-133)

high

indexBuilder と同様に、ydocState が空のまま保存されています。これにより、スキーマページをエディタで開いた際に内容が表示されません。プレーンテキストから ydocState を生成して保存するように改善してください。

src/components/layout/ImageCreateDialog.tsx (300-303)

high

画像のアップロード(uploadImage)が OCR や画像解析の前に行われています。もし解析処理でエラーが発生したり、ユーザーが途中でキャンセルしたりした場合、アップロード済みの画像が S3 上で孤児(Orphan)になってしまいます。解析処理を先に実行し、成功した場合のみ最後にアップロードを行う順序に変更することを推奨します。

server/api/src/routes/ingest.ts (371-384)

medium

sources テーブルへの挿入時に一意制約(uq_sources_owner_url_hash)による衝突が発生する可能性があります。onConflictDoUpdate を使用して、衝突時に既存のレコードを更新しつつその ID を取得するように変更することを検討してください。これにより、並行リクエスト時の 500 エラーを回避できます。

server/api/src/routes/ingest.ts (232-233)

medium

トークン使用量の計算が文字数ベースの概算になっています。IngestLlmDrivercallProvider から返される実際の usage を返すようにリファクタリングし、正確なトークン数を recordUsage に渡すように改善することを検討してください。

src/components/layout/Header/NavigationMenu.tsx (31-34)

medium

デスクトップ版のナビゲーションメニューから AI へのリンクが消えています。サイドバーが廃止されたため、このメニューに AI/ai)を追加しないと、デスクトップユーザーが過去の会話履歴一覧にアクセスする手段が失われてしまいます。

const NAV_ENTRIES: readonly NavEntry[] = [
  { path: "/home", icon: Home, i18nKey: "nav.home", exact: true },
  { path: "/notes", icon: FileText, i18nKey: "nav.notes" },
  { path: "/ai", icon: Sparkles, i18nKey: "nav.ai" },
] as const;

server/api/src/services/lintEngine/rules/titleSimilar.ts (61-64)

medium

全ページ(N件)に対して二重ループで Levenshtein 距離を計算しているため、計算量が O(N^2) となっています。ページ数が多いユーザー(数千件以上)の場合、この処理がイベントループを長時間ブロックし、API の応答性に悪影響を与える可能性があります。頭文字によるバケツ分けや、類似度の高い候補を絞り込むためのインデックス活用など、最適化を検討してください。

@otomatty otomatty merged commit cfd6c42 into fix/lint-engine-rules Apr 19, 2026
18 checks passed
@otomatty otomatty deleted the claude/address-pr-646-comments-VqCIx branch April 19, 2026 23:34
otomatty added a commit that referenced this pull request Apr 20, 2026
…, surface exact-title dups (#646)

* fix(lint-engine): normalize fact values, count distinct ghost sources, surface exact-title dups

PR #636 review コメントへの対応 (lint engine の偽陽性・偽陰性修正)。

- rules/conflict.ts: 抽出時に値を正規化する。日付は YYYY-M-D に、数値はカンマ・
  空白を除去してから格納。これで `2026-04-19` と `2026/4/19`、`1,000円` と
  `1000円` のような format-only の差分が誤って conflict として通知されなくなる
- rules/ghostMany.ts: count(*) と array_agg を `DISTINCT source_page_id` に変更。
  同一ページ内で同じ link_text が複数回現れるケースを別ソースとして二重計上して
  しまい閾値を超えていた偽陽性を解消。HAVING も distinct count で判定
- rules/titleSimilar.ts: 完全一致タイトルを「他で処理する」というコメントだけで
  握り潰していたが、実際にはどこも見ておらず exact-dup が黙って素通りしていた。
  distance=0 を最も強い title_similar finding として正規に通知するよう修正
- rules/conflict.test.ts に正規化ケースのテストを追加

Made-with: Cursor

* fix(lint-engine): support space-grouped thousands, canonical decimals, exact-dup severity (#650)

PR #646 のレビュー対応。

- conflict.ts: `NUMBER_PATTERN` に `(?: \d{3})*` を追加し、空白区切りの
  thousand grouping (`1 000 円` / `1 000 000 円`) を取りこぼさない。
  3 桁ずつのみ許容することで `100 3人` のような偽マッチは回避する。
  (coderabbitai #3107605897)

- conflict.ts: `normalizeNumberValue` を単位分離 + `parseFloat` ベースに
  書き換え、末尾の小数ゼロ (`1000` vs `1000.0`, `3.776` vs `3.7760`) を
  canonical 形式に畳む。
  (gemini-code-assist #3107607604)

- titleSimilar.ts: 完全一致 (`distance=0`) は重複ページ由来のリンク曖昧
  さを生むため `severity: "warn"` に上げ、近似一致 (`distance>0`) は
  従来どおり `info` のまま。finding 内の suggestion 文言も分岐させる。
  (gemini-code-assist #3107607603)

- conflict.test.ts: 空白区切り、百万単位、末尾 `.0`、小数末尾ゼロを
  同値として正規化するテストを追加 (vitest 29/29 passing)。

https://claude.ai/code/session_012J9gqETyditCqdU6xA6z87

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants