common : preserve original Gemma 4 tool responses even when JSON-like#21522
common : preserve original Gemma 4 tool responses even when JSON-like#21522kiwixz wants to merge 1 commit intoggml-org:masterfrom
Conversation
|
Which coding agent are you using? This might be the best thing to do, but it does have negative implications for clients that send structured tool output. |
|
I'm developing my own thing. I found that text works better than structured things at least for simple tools. It's too bad that OpenAI API doesn't support just passing a JSON object directly in there; would be much more elegant than forcing it to a string and then guessing what it was... |
|
Many read tools prepend the line number to each line, which would make this get handled as a string. Here is my concern: the model may in-context learn to start producing JSON instead of its native structure. We see this happen in other models where we manipulate the input in a way it wasn't trained. |
Overview
Before this PR, llama.cpp Gemma 4 specialized parser (#21418) would attempt to parse the string-typed tool responses as JSON; and convert it to an object if successful. It would then be minified by the template:
This means that anything that looks like JSON would get unexpectedly altered.
I encountered this with Gemma 4 failing to edit a JSON file; it was getting confused as the inconsistent line numbers and indentations between what it read and what it attempted to write/edit.
Additional information
Requirements