You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
GLM-5.3-Flash on main: export fails and prefix-cache restores fail with device error 72 #272
GLM-5.3-Flash can't be served from main (0572fd1). There are two independent problems.
1. Export fails while compiling, on every rank:
torch._dynamo.exc.UserError: Could not extract specialized integer from data-dependent expression u51
... block_fp8.py _project -> plan_from_handle(handle)
Since 4a24d78, _project is traced and looks up its plan from a handle that preparation marks as unbacked.
2. Every prefix-cache restore fails with prepared model reported device error 72 once export works. The first request succeeds and every later request fails.
Merge 2b38bf9 kept #165's two-page trailing regions but restored the older trailing_groups[region.nbytes] key (state_transfer.py:273). That key is two pages, so each shifted MTP draft copy writes two pages at the wrong offset. The copy overwrites last_hidden and consumed_tokens, and the next restore fails validation. Only GLM-5.3 passes trailing pages, so no other model is affected.
Activity