Skip to content

feat(harness): 添加实际能力诊断与五类加载采样 - #66

Open
xu91102 wants to merge 3 commits into
mainfrom
codex/capability-loading
Open

xu91102 wants to merge 3 commits into
mainfrom
codex/capability-loading

Conversation

@xu91102

@xu91102 xu91102 commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

背景

现有 cew verify 只能校验分发源码,不能发现“登记表相同、已安装技能正文不同”的情况,也没有单条任务实际读入材料的统计入口。

核心改动

  • 新增只读 cew doctor:比较技能正文、登记表和直接引用,沿用项目优先及宿主规则回退;区分发现、静态就绪、阻塞和未确认,重复安装只报告候选,不覆盖个人文件。
  • 新增 cew loading:从 Codex CLI JSONL 的完整正文工具输出统计文件指纹、读入量和重复读取;不把文件名提及当作读取,不把字符估算当作实测 Token。
  • 建立普通问答、小修复、正式设计、执行验收、建立验收能力五类自然语言采样用例及显式运行工具,记录首轮真实结果。未调整技能路由或生命周期协议。
  • 为诊断增加 CI 回归;安装器按既有精确清单排除依赖分发包的诊断脚本,保留用户目录中的其他文件。

验证

  • npm run verify:通过。
  • npm run test:diagnostics:21 项通过,包括共享用户技能目录(含自定义 CODEX_HOME)、项目技能路径归一、个人同级脚本、缺失同级依赖保护、正文漂移、引用回退、重复候选、路由代码块依赖、重复/重叠读取及超时后退出码为 0 的回归。
  • test:manifest、test:invocation、test:continuation、test:install-rules、test:plugins:通过。
  • 打包并在独立目录安装生产依赖后,CLI verify、doctor、loading 冒烟通过;git diff --check 通过。
  • 独立只读审查:Standards / Spec 通过。
  • 真实采样:普通问答、小修复、执行验收完整结束;正式设计与验收建设在三分钟上限超时,明确保留为 BLOCKED,不计通过。原始轨迹仅保留在本地忽略目录,仓库只提交统计及限制说明。

风险与回滚

STATIC_READY 只覆盖可确认的直接本地引用,不证明宿主选择或所有分支可执行。加载统计不覆盖宿主自动注入、部分及变形读取;本轮没有 Token 节省结论。Windows 实机、安装升级实机、Claude 轨迹及桌面 App 验收尚未覆盖。

新增入口可随本提交回退移除;诊断不会修复或覆盖个人安装文件。

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T01:48:50.129533Z 24d1c73 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

xu91102 commented Nov 2, 2026

Copy link
Copy Markdown
Owner Author

@codex review

请审查当前提交 50a90cc。重点核查:只读诊断的正文/引用/候选判定是否准确;项目与宿主规则回退及个人文件保护;加载统计是否只据完整工具输出且避免重复计数;错误、超时及分发包/安装目录边界。

五类真实采样为三类完整、两类超时;不应据此认定所有任务已通过或已节省 Token。

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50a90ccb37

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/diagnostics/loading.js Resolved
Comment thread scripts/diagnostics/capabilities.js Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ee38f1d8c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/diagnostics/sample-loading.js Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant