Auto-retry startup OOM once with a reduced memory footprint

[velaraptor-runpod]

Why

CUDA OOM at init is the dominant worker-vllm first-request failure class — and it persisted after the v2.27.1 rollback . Drivers: v0.29 budgets the KV pool as requested − non-KV − cudagraph_memory_estimate (graphs reserved up front — see vllm#57475; the proper fix, vllm#51590, is still open) and the max-num-batched-tokens default doubled in v0.28. A boot that OOMs on defaults is answered with advice… and then crash-loops on the exact same footprint. One relaxed retry can genuinely fit where the defaults could not (upstream's own issue shows --enforce-eager booting configs that OOM with graphs).

What

A startup OOM now gets one reduced-footprint relaunch, mirroring the vanished-revision relaunch from #343:

• Trigger: startup_errors.memory_shortfall() — any of the three memory wordings (torch.OutOfMemoryError / "CUDA out of memory", "No available memory for the cache blocks", "KV cache too small").
• Relaunch env: ENFORCE_EAGER=true (drops the up-front graph reserve) and MAX_NUM_BATCHED_TOKENS=8192 (halves peak activation).
• Per-knob relaxation, never blind override: an explicit ENFORCE_EAGER value is overridden but loudly logged; a user-set token budget ≤ 8192 is kept; placeholder MAX_NUM_BATCHED_TOKENS=0 counts as unset (matches args_builder's ZERO_MEANS_UNSET). When both knobs are already at/over their recovery value the retry is skipped as pointless and jobs get the existing advice immediately.
• If the second launch OOMs too, jobs are answered with the cause — the message now states the retry already ran and lists only the remaining real fixes (smaller/quantized checkpoint, larger GPU or TENSOR_PARALLEL_SIZE, lower MAX_MODEL_LEN, KV_CACHE_DTYPE=auto on Ampere/Ada).
• Bounded: at most 3 launches total (revision retry and OOM retry are independent single-use budgets). Every other failure mode and the happy path are byte-for-byte unchanged — the recovery code only runs after a startup failure that would previously have been terminal.

Intent: on marginal models (24/48 GB cards) the endpoint comes up slowly-but-serving instead of crash-looping or dead. Degraded-mode intent is loudly logged (no response-metadata channel exists for it today).

Tests

pytest tests/ — 92 passed (13 new/updated):

• tests/test_main_oom_retry.py (new): retry-that-succeeds, never-a-third, per-knob relaxation, explicit-config respect (ENFORCE_EAGER=true + budget pre-tightened → no retry), KV wordings share the class, revision+OOM budgets compose (3 launches max).
• tests/test_startup_errors.py: predicate coverage for all three memory wordings; the post-retry message says what ran and omits the already-applied knobs.
• tests/test_main_revision_retry.py: its old "other fatal failures never relaunch" case used an OOM output — now a gated-repo error, matching the new policy.

[promptless]

Promptless documentation updates

• Document vLLM worker startup error handling in Serverless troubleshooting now explains the one-time out-of-memory retry at startup: which settings it changes, when it is skipped, and what to try if it also fails.