Skip to content

test: validate documented JSON responses in both tools references #601

Open

Description

@yeongseon

Source-observed documentation test gap

docs/TOOLS.md and its Korean equivalent each contain 13 fenced json examples for MCP tool results. Existing tests/test_docs_alignment.py checks that every tool is mentioned, but does not parse every documented example into JSON, so a typo or drift in example syntax can remain until a user copies it.

This is separate from #580, which owns README client configuration snippets, not the two per-tool result reference pages.

Small task

  • In existing offline docs-alignment tests, extract the actual fenced json blocks from the two TOOLS pages and check json.loads succeeds on each complete result example. Skip unrelated Python/shell/SQL fences.
  • If an intentionally illustrative fragment is not valid JSON, clearly mark and exclude that specific fragment with a reviewed reason, rather than weakening the entire check.
  • Optionally verify each example heading/tool name remains available in English and Korean, without requiring identical translated prose or ordering.

Acceptance

  • A deliberately malformed JSON result block in either language fails the focused test; all valid current examples pass.
  • False positives are avoided for Markdown prose and non-JSON code blocks; no blanket Markdown parser/dependency is introduced.
  • Run the focused test and existing docs-related checks; keep tool registration, schemas and runtime unchanged.

A suitable small contributor task: one test file, no database or client installation required.

Activity

  1. yeongseon commented on Nov 21, 2026

    @yeongseon
    ContributorAuthor

    Triage reviewed (2026-11-21): Ready for human-led contributors. Scope is one offline docs-reference JSON validation test against docs/TOOLS.md and docs/ko/TOOLS.md; the existing #580 owns README client-config snippets and does not overlap. No CUBRID server required. Validate via python -m pytest tests/test_docs_alignment.py -q and the repository's normal check/test CI. Confirm the JSON fences exist and report any intentionally illustrative fragments without silently accepting malformed examples. No source API or documentation content restructure. An unassigned good-first issue may be implemented by human-led contributors; maintainer-operated autonomous agents need separate issue-specific authorization and assignment under CONTRIBUTING.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueSmallhelp wantedMaintainers welcome an external PR and will answer questionspriority: lowNice to havesize: SSmall changetestingTest coverage or test fixes

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions