Skip to content

fix: raise default VLLM_STARTUP_TIMEOUT to 1800s and expose it in the hub env schema - #354

Merged
velaraptor-runpod merged 2 commits into
mainfrom
fix/vllm-startup-timeout-default-1800
Oct 7, 2026
Merged

velaraptor-runpod merged 2 commits into
mainfrom
fix/vllm-startup-timeout-default-1800

Conversation

@velaraptor-runpod

Copy link
Copy Markdown
Contributor

What

  • Raises the default startup health-check deadline in src/main.py from 1200s to 1800s (VLLM_STARTUP_TIMEOUT override unchanged).
  • Adds VLLM_STARTUP_TIMEOUT to the .runpod/hub.json env schema (advanced, default 1800) so hub/console deploys surface it; today the knob exists in code but is invisible to users.
  • Keeps docs/configuration.md in sync.

Why

On 2026-10-06 a 4×H200 worker serving DeepSeek-V4.1-Flash (475 GB, fp8_ds_mla) had its weight load stretched to ~13 min by same-host contention from overspawned sibling workers. The 1200s watchdog fired and SIGTERM'd the vLLM process 13 seconds before it printed "Application startup complete", forcing a full container restart and another multi-minute engine bring-up. For 400 GB+ models, 1200s leaves no headroom for variance in page-cache warmth, host contention, or disk pressure — and killing an actively-loading backend wastes the entire load it just paid for.

1800s covers the observed worst case (789s weight load + graph capture + warmup) with margin, while still bounding genuinely stuck starts.

Not in this PR

Follow-up worth a separate issue: make the watchdog progress-aware (don't SIGTERM a backend still actively loading weights) instead of purely wall-clock.

Testing

pytest tests — 142 passed. hub.json parses and matches the sibling entry shape.

@velaraptor-runpod
velaraptor-runpod merged commit b7212ac into main Oct 7, 2026
11 checks passed
@velaraptor-runpod
velaraptor-runpod deleted the fix/vllm-startup-timeout-default-1800 branch October 7, 2026 01:15
@promptless

promptless Bot commented Oct 7, 2026

Copy link
Copy Markdown

Promptless documentation updates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants