Skip to content

Add MCP Lens Schema Audit to agent-specific evaluation - #79

Open
labmimors wants to merge 1 commit into
benchflow-ai:mainfrom
labmimors:add-mcp-lens-schema-audit
Open

Add MCP Lens Schema Audit to agent-specific evaluation#79
labmimors wants to merge 1 commit into
benchflow-ai:mainfrom
labmimors:add-mcp-lens-schema-audit

Conversation

@labmimors

Copy link
Copy Markdown

Why this belongs

This adds one deterministic GitHub Action to §9 · Agent-specific evaluation. Given a checked-in model-facing MCP tool payload, it reports tool count and Tool Schema JSON UTF-8 bytes and can fail CI on explicit max-tools or max-schema-bytes thresholds.

This is deliberately a component-level check for the tool surface presented to an agent, not an end-to-end task or agent-quality evaluation.

Show your work

  • The Action contract fixes the Node 24 runtime, inputs, outputs, and budget behavior.
  • The implementation uses no model call, API key, or network request.
  • The public Action tests cover accepted payload shapes, numeric outputs, threshold failures, and Step Summary behavior. I reproduced all 9 tests in a fresh checkout before opening this PR.

Verification

  • Confirmed both canonical links return HTTP 200.
  • Checked this repository plus open/closed GitHub issues and PRs for dsh-mcp-lens, MCP Lens, and the canonical repository URL; no existing submission was found.
  • Ran git diff --check; the content change is one README entry.

Disclosure

I am the maintainer of labmimors/dsh-mcp-lens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants