feat: validate workflow model references before submitting to ComfyUI

[lukepiette]

What

Adds a pre-flight validation step that checks every model file a workflow references against ComfyUI's /object_info before the workflow is submitted — so a missing checkpoint/LoRA/VAE/CLIP/UNet (or the literal '__list__' placeholder some clients send) fails fast with a clear, actionable error instead of a cold start followed by a cryptic crash.

Before / After

Before: the worker cold-starts (often ~20 min pulling models), the workflow is submitted, and ComfyUI rejects it with Value not in list: ckpt_name, Invalid image file: /runpod-volume/..., or a bare prompt_outputs_failed_validation. The existing 400-handling in queue_workflow() enriched only the checkpoint case (CheckpointLoaderSimple), missing LoRAs, VAE, CLIP, UNet, upscale models, and the '__list__' placeholder entirely.

After: the handler validates the workflow right after image upload and before queue_workflow(), and returns one error listing every missing file, its model type, and the expected network-volume directory:

Workflow validation failed — the following referenced files are not available on this worker:
• Node 4 (CheckpointLoaderSimple.ckpt_name): 'sdxl_typo.safetensors' not found in checkpoints. Expected at /runpod-volume/models/checkpoints/sdxl_typo.safetensors. Available checkpoints: 'sd_xl_base_1.0.safetensors', …
• Node 7 (LoraLoader.lora_name): 'missing_lora.safetensors' not found in loras. Expected at /runpod-volume/models/loras/missing_lora.safetensors. …

Upload the missing file(s) to the matching directory on your network volume, or update the workflow to use one of the available files.

The '__list__' placeholder gets its own message explaining the client sent a UI default that was never replaced with a real filename.

Design decisions

• Validates against /object_info (what ComfyUI actually has registered — authoritative, covers baked-in and volume models alike). The expected-location hint maps model types to the volume layout from src/extra_model_paths.yaml (text encoders → models/clip/, diffusion models → models/unet/).
• Loader coverage per the node→directory mapping in AGENTS.md: CheckpointLoaderSimple, LoraLoader, VAELoader, DualCLIPLoader/TripleCLIPLoader, UNETLoader/UnetLoaderGGUF/Hy3DModelLoader, UpscaleModelLoader, plus LoadImage (the Invalid image file case).
• Collects all misses into one error rather than failing on the first.
• Case-sensitive matching (mirroring ComfyUI's own list validation), with a "Did you mean 'X'? (filenames are case-sensitive)" hint when a file differs only in case.
• Subfolder-relative names (subdir/file.safetensors) compare against the same relative form /object_info returns.
• Fails open on infra errors: if /object_info can't be fetched, the pre-flight is skipped and the existing queue_workflow() 400-enrichment remains as the backstop — a network blip is never misreported as a user error. Loader node types not registered in the running ComfyUI build are skipped rather than guessed at. Annotated LoadImage refs (file.png [output]) are also left to ComfyUI.
• get_available_models() is generalized from checkpoint-only to all loader types; queue_workflow()'s existing backstop behavior is unchanged.

Test suite repair (pre-existing breakage)

tests/test_handler.py did not import on main (from src import handler — handler.py lives at the repo root) and several tests targeted functions removed in #118 (base64_encode, process_output_images, the old urllib-based queue_workflow/get_history). CI's test job is currently an echo placeholder, so this had gone unnoticed. This PR repairs the suite: fixed imports, updated stale tests to the current signatures, removed tests for functions that no longer exist, and added coverage for the new pre-flight.

Validation

python -m unittest discover tests -v → 28 tests, all passing (on main the suite fails at import). New coverage:

• present file (incl. subfolder-relative) → passes validation, workflow reaches queue_workflow() unchanged
• missing checkpoint → named, with type + expected volume dir + available list
• missing LoRA + VAE + CLIP in one workflow → all reported together
• '__list__' → specific placeholder message
• case-only mismatch → did-you-mean hint
• linked inputs (["7", 0]) and unregistered loader types → skipped
• /object_info unreachable → fails open (no error)
• handler-level ordering: pre-flight failure short-circuits before queue_workflow() is called

Note per AGENTS.md: handler.py is ADDed into the image at build time (no hot reload) — logic validated via unit tests as recommended there.

🤖 Generated with Claude Code

[lukepiette]

Real-GPU validation (Runpod secure-cloud A40, real ComfyUI server with SD1.5-fp16 on disk):

• Unit suite: Ran 28 tests … OK.
• Live pre-flight against the real /object_info: workflow referencing a missing checkpoint and a missing LoRA → one error in 0.19s naming both files, their model types, expected volume paths (/runpod-volume/models/checkpoints/…, /runpod-volume/models/loras/…), and the available alternatives.
• '__list__' placeholder → the specific "client sent a UI default that was never replaced with an actual file" message, with the available checkpoints listed.
• Present checkpoint → validation returns None, and a full handler() run generated a real 512×512 image in 6.5s (1 image, base64) — no behavior change on the happy path.
• handler() with a missing model → blocked in 0.26s, before any ComfyUI submit, with the full actionable error.

🤖 Generated with Claude Code

[promptless]

Promptless documentation updates

• Add ComfyUI missing-model troubleshooting entry to Serverless troubleshooting documents the new Workflow validation failed pre-flight error and how to resolve it — the /runpod-volume/models/ directory each model type belongs in, case-sensitive filename matching, the __list__ placeholder case, and the fail-open behavior when ComfyUI's model list is unreachable.