Repository navigation
feat: update to 0.19.1 - #287
Merged
Merged
Conversation
- Bump vllm[flashinfer] to 0.19.1 in Dockerfile - Add OpenAIServingRender (new required dependency in 0.19.x serving layer) - Pass openai_serving_render to all four serving class constructors - Remove log_error_stack param (removed upstream in 0.19.x) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…zero adapters Fixes FDE-194. Previously a malformed LORA_MODULES value was swallowed at info level and the engine would start with no LoRA adapters, causing 500s on any request using an adapter model name (e.g. npc-sim-*). Changes: - Log at error level when LORA_MODULES cannot be parsed as JSON - Log at error level when individual adapter dicts fail LoRAModulePath validation - Log a final error when all adapters fail to load so the cause is obvious - Accept a single adapter dict (not just an array) for convenience - Return early when LORA_MODULES is unset to skip unnecessary parsing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…asing Fixes FDE-174. Some model stores (e.g. RunPod pre-cached network volumes) normalize repo IDs to lowercase. HuggingFace Hub caches using the original casing, so MODEL_NAME=Qwen/Qwen2.5-Coder-32B-Instruct-AWQ would miss a cache stored as models--qwen--qwen2.5-coder-32b-instruct-awq/ and attempt a redundant download that fails on limited container storage. If the exact-case HF cache directory is absent but a lowercase variant exists, the latest snapshot path is returned directly so vLLM loads from disk. Absolute paths and models with no lowercase cache are unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
velaraptor-runpod
requested review from
Madiator2011Work,
TimPietruskyRunPod and
Yhlong00
April 30, 2026 23:39
Yhlong00
approved these changes
May 1, 2026
Contributor
Author
|
ok tested. |
…gments to True for OOM, for hub defaults
Contributor
Author
jhcipar
approved these changes
May 1, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.