Skip to content

feat: CON-1146: Add more graceful error handling - #337

Merged
velaraptor-runpod merged 1 commit into
mainfrom
owen/con-1146-serverless-oom-vllm-workers-crash-loop-on-startup-cuda-oom
Sep 10, 2026
Merged

velaraptor-runpod merged 1 commit into
mainfrom
owen/con-1146-serverless-oom-vllm-workers-crash-loop-on-startup-cuda-oom

Conversation

@OWCramer

@OWCramer OWCramer commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Currently when the worker runs into an issue such as an OOM the process silently dies, the worker goes unhealthy, and the user gets confused. The changes in this PR catch those error and send the error to the user via the standard output. So instead of getting confused and tryign to filter through logs, the request instead comes back with an OOM error.

This approach mirrors implementations from this org's vllm-omni repo and ollama repo.

@velaraptor-runpod
velaraptor-runpod self-requested a review September 8, 2026 19:00
velaraptor-runpod added a commit that referenced this pull request Sep 10, 2026
Cloudflare now bans the default Python-urllib UA with HTTP 403
(error code 1010), which broke the serverless e2e job for every PR
today (this one and #337). A browser UA passes.
@velaraptor-runpod

Copy link
Copy Markdown
Contributor

Can you add this header in the serveless_e2e_test. You can look at this PR, or better just merge that branch here: #338.

Just so we properly test it.

    # Cloudflare in front of rest.runpod.io bans the default Python-urllib UA
    # (HTTP 403, error code 1010, first seen 2026-09-10). A browser UA is not
    # banned; the value just needs to look like one.
    req.add_header(
        "User-Agent",
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/126.0 Safari/537.36",
    )

@velaraptor-runpod velaraptor-runpod left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refer to comment above.

@OWCramer
OWCramer force-pushed the owen/con-1146-serverless-oom-vllm-workers-crash-loop-on-startup-cuda-oom branch from 34e41bb to 3de69dc Compare September 10, 2026 20:18
@velaraptor-runpod
velaraptor-runpod merged commit 76054c2 into main Sep 10, 2026
11 checks passed
@velaraptor-runpod
velaraptor-runpod deleted the owen/con-1146-serverless-oom-vllm-workers-crash-loop-on-startup-cuda-oom branch September 10, 2026 21:04
@promptless

promptless Bot commented Sep 10, 2026

Copy link
Copy Markdown

Promptless documentation updates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants