Skip to content

[Regression] Qwen3.5 saved checkpoint is not correct with save_pretrained API since version 5.4.0 #45216

@xin3he

Description

@xin3he

System Info

transformers == 5.3.0 works well
transformers ==5.4.0 returns Unexpected model.language_model.language_model.language_model.layers.7.self_attn.v_proj.weight in loaded safetensors file

Who can help?

@zucchini-nlp

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

import transformers

name = "Qwen/Qwen3.5-0.8B"
model = transformers.Qwen3_5ForConditionalGeneration.from_pretrained(name, trust_remote_code=True)
model.save_pretrained("./qwen-35")

from safetensors.torch import save_file, load_file

loaded = load_file("./qwen-35/model.safetensors")
assert not 'model.language_model.language_model.language_model.layers.7.self_attn.v_proj.weight' in loaded, "Unexpected model.language_model.language_model.language_model.layers.7.self_attn.v_proj.weight in loaded safetensors file"

Expected behavior

assert pass

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions