Add GPT-OSS tool calling support

[qgallouedec]

• Add response schema for GPT-OSS (done by Claude)
• Add gptoss.jinja template for identity matching in add_response_schema
• Add GPT-OSS to TestAddResponseSchema and TestParseResponse test parametrizations
• Add GPT-OSS to supported models in agent training docs

Part of #5460

Warning

Requires/contains #5459

---

Note

Medium Risk
Introduces a new GPT-OSS response parsing schema and updates GRPO tool-suffix tokenization to depend on real tool names, which could affect tool-call loop formatting and parsing across models if templates behave differently.

Overview
Adds GPT-OSS tool-calling support by introducing a new gptoss.jinja identity template and a corresponding gptoss_schema, and wiring both into add_response_schema so parse_response can extract content and single tool calls from GPT-OSS outputs.

Updates GRPO tool-call suffix extraction in both GRPOTrainer and AsyncRolloutWorker to build the dummy conversation using the actual tool name (not "dummy"), matching templates (like GPT-OSS) that derive tool-response headers from the preceding tool call.

Extends response-schema/parsing tests to include a tiny-GptOssForCausalLM tokenizer with model-specific skips/expectations, and documents GPT-OSS as a supported agent-training model in grpo_trainer.md.

^{Reviewed by Cursor Bugbot for commit 392dece. Bugbot is set up for automated code reviews on this repo. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b18e39efd6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[qgallouedec]

cc @Rocketknight1 for the schema

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 450b9ef. Configure here.}

[cursor]

Prefix-preserving check diverges from suffix extraction construction

Low Severity

is_chat_template_prefix_preserving still uses a hardcoded "dummy" tool name, while _get_tool_suffix_ids was changed to use the real tool name via tool_messages[0]["name"]. The comment on line 350 explicitly states "Use the same dummy messages as _get_tool_suffix_ids", but the constructions now differ. For GPT-OSS, the tool name is embedded in the rendered text (e.g. to=functions.NAME), so the validation function is no longer testing the exact property that _get_tool_suffix_ids relies on.

Additional Locations (2)

• trl/trainer/grpo_trainer.py#L1418-L1419
• trl/experimental/async_grpo/async_rollout_worker.py#L559-L560

 

^{Reviewed by Cursor Bugbot for commit 450b9ef. Configure here.}