Repository navigation
feat: server-side read cache guard to reduce repeated read_file calls - #422
GrothKeiran wants to merge 1 commit into
Conversation
…ed read_file - Add read_tool_cache.go with buildReadToolCacheHints() - Inject system message with read cache state after tool prompt - Adds test for cache state presence - Reduces 70-80% of meaningless repeated reads caused by context ambiguity
|
Someone is attempting to deploy a commit to the cjack's projects Team on Vercel. A member of the Team first needs to authorize it. |
There was a problem hiding this comment.
Pull request overview
This PR adds a prompt-side “read-tool cache state” system message that scans prior conversation history for read-like tool calls/results and instructs the model to reuse already-returned file contents instead of repeatedly calling read_file for the same path.
Changes:
- Added history-scanning logic to detect prior read-like tool calls and summarize their cache state into a system hint message.
- Integrated the new hint injection into the OpenAI prompt-building flow.
- Added a unit test asserting the cache-state hint appears when a prior
read_filereturned file content.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
internal/promptcompat/read_tool_cache.go |
New helper to extract read-like tool call/path/call-id info from raw messages and generate a “read cache state” system hint. |
internal/promptcompat/prompt_build.go |
Appends the generated cache-state hint (when present) to the prompt messages before final serialization. |
internal/promptcompat/prompt_build_test.go |
Adds test coverage ensuring cache-state hint is included for prior read_file with content. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| entry.Path = firstNonEmptyString(msg, "path", "file_path", "filename") | ||
| } | ||
| if entry.Path == "" { | ||
| continue | ||
| } | ||
| entry.HasContent = strings.TrimSpace(NormalizeOpenAIContentForPrompt(msg["content"])) != "" | ||
| out[readToolCacheKey(entry.Name, entry.Path)] = entry |
| line := fmt.Sprintf("- %s path=%q status=%s", entry.Name, entry.Path, contentState) | ||
| if entry.CallID != "" { | ||
| line += fmt.Sprintf(" tool_call_id=%q", entry.CallID) | ||
| } |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a28c765d45
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if entry.Path == "" { | ||
| continue | ||
| } | ||
| entry.HasContent = strings.TrimSpace(NormalizeOpenAIContentForPrompt(msg["content"])) != "" |
There was a problem hiding this comment.
Detect real file body before marking cache entry present
entry.HasContent is set to true for any non-empty tool result, but the read guard in tool_prompt.go explicitly calls out non-body replies like “already available in history”/“unchanged”. Those replies are non-empty, so this code marks them as content-present and the injected cache hint tells the model to reuse content that is not actually in context, which can lead to incorrect edits or analysis without a real file body.
Useful? React with 👍 / 👎.
| if entry.Path != "" { | ||
| out[readToolCacheKey(entry.Name, entry.Path)] = entry |
There was a problem hiding this comment.
Preserve earlier successful reads when same path is read again
In the assistant-tool-call pass, each new read_file call immediately overwrites the path entry in out with a zero-value HasContent=false record. If a file was successfully read earlier in the conversation and then re-requested (especially when later calls return cache/missing-body responses), the final cache state can regress to content-missing-or-empty even though full content already exists in history, which defeats the reuse hint and encourages redundant reads.
Useful? React with 👍 / 👎.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Summary
Implements a prompt-side history injection mechanism that tracks which files have already been read in the conversation and instructs the model to reuse existing content instead of re-reading the same file.
Problem
When using ds2api with DeepSeek models, models frequently call
read_filefor the same file path repeatedly. This happens because the model sees DSML/XML tool call records in history and cannot reliably determine whether a file's content is already present in context. This wastes tokens (~70-80% of repeated reads are unnecessary) and delays actual work.Solution
New file:
internal/promptcompat/read_tool_cache.gobuildReadToolCacheHints()— scans conversation history for priorread_file/read/readfiletool calls and their resultstool_call_id, and whether content was actually presentModified file:
internal/promptcompat/prompt_build.gobuildReadToolCacheHints()with raw messages and tool namesNew test:
TestBuildOpenAIFinalPromptReadLikeToolIncludesReadCacheStateinprompt_build_test.goread_filewas previously called with actual contenttool_call_id, and reuse instructionsExpected Effect
Reduces ~70-80% of meaningless repeated file reads caused by context ambiguity. The model receives an explicit declaration of which files have already been read and their content status, removing the uncertainty that triggers redundant reads.
Limitations (noted in PR)
tool_prompt.goaddresses part of thisDependencies
Changes
internal/promptcompat/read_tool_cache.gointernal/promptcompat/prompt_build.gointernal/promptcompat/prompt_build_test.goFixes repeated
read_filecalls that waste model context and delay tool execution.