Skip to content

fix: name the failure when a model cannot be run - #6

Open
OWCramer wants to merge 4 commits into
mainfrom
fix/unrunnable-model-errors
Open

OWCramer wants to merge 4 commits into
mainfrom
fix/unrunnable-model-errors

Conversation

@OWCramer

@OWCramer OWCramer commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Two failures seen on dev today that the worker handled badly, plus a doc correction.

A quantised model crash-looped instead of reporting itself

ovedrive/Qwen-Image-Edit-2511-4bit exits at startup with:

factory.py:95   return DiffusionBitsAndBytesConfig(**kw)
TypeError: DiffusionBitsAndBytesConfig.__init__() got an unexpected keyword argument '_load_in_4bit'

BitsAndBytesConfig.to_dict() emits the private backing fields next to the public ones — that repo's transformer/config.json contains _load_in_4bit, _load_in_8bit, load_in_4bit and load_in_8bit — and the engine splats the whole dict into a dataclass declaring only the public names. The repo is correctly formed; this will hit every bitsandbytes-quantised diffusers repo on v0.28.0.

classify returned None for it, so the worker exited 1 and respawned every 30s. It now answers with the cause, since no restart can fix a config the loader rejects.

A runtime OOM never reached the caller

Wan-AI/Wan2.2-I2V-A14B-Diffusers keeps both transformers resident: 126GB of fp32 on disk loads into 64.21 GiB, and on an 80GB card the first request dies in the VAE encode with 1.55 GiB free. The engine is healthy, so classify — which only runs before the server comes up — never sees it.

The proxy returned the raw CUDA text, several paragraphs of allocator state, and the platform then rejected the result:

Failed to return job results. | 400, message='Bad Request', url='.../job-done/...'

So the caller got nothing at all. classify_runtime_error now replaces an OOM response with one sentence naming the fix. It sits beside classify because the output is the same kind of thing at a different moment; anything it does not recognise passes through untouched.

I have not confirmed the 400 was caused by the payload size — that is inference from the shape of it. Shortening the message is worth doing either way.

README

The architecture count said "~72"; scripts/omni_supported_archs.json has 48. Corrected to 48, which matters more now that the console reads that file at runtime rather than keeping its own copy.


Three tests added, 43 passing. I checked the previous classifier against both inputs to confirm the gaps were real: classify returned None for the quantisation error, and classify_runtime_error did not exist.

Not reproduced on a GPU — both classifiers are written against log output captured from the real dev failures.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant