fix: compatibility with transformers >= 5 and support non-llama models in chat plugin#171
Merged
Merged
Conversation
…s in chat plugin 1. Catcher: add __getattr__ to proxy wrapped module attributes (e.g. attention_type for Qwen3), fixing AttributeError during input hijacking. 2. base.py: add _no_init_weights() fallback for transformers >= 5 where no_init_weights was removed. 3. inference.py: enable chat_loop_v3 for non-llama models with apply_chat_template support.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
quant_frame_base.py: Add
__getattr__toCatcherto proxy wrapped module attributes (e.g.attention_typefor Qwen3), fixingAttributeErrorduring input hijacking.base.py: Add
_no_init_weights()fallback context manager for transformers >= 5 wheretransformers.modeling_utils.no_init_weightswas removed.inference.py: Enable
chat_loop_v3for non-llama models that haveapply_chat_templatesupport.