fix(deployments): tolerate transient gateway 5xx during deploy status polling - #212
Conversation
…ring deploy A gateway 502 on the deployment status endpoint mid-poll made 'deployments deploy' fail with 'OSS request failed: 502' even though the deployment itself went on to reach READY. Treat 5xx CLIErrors from the status/sync requests as transient (like network-level fetch errors) and keep polling; deployment-failure errors (no statusCode) and 4xx responses still fail fast. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Greptile SummaryThis PR makes deployment polling resilient to transient status-read failures.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| src/commands/deployments/deploy.ts | Updates deployment polling to retry transient HTTP and network failures, retain timeout context, and expose the polling helper and constants. |
| src/commands/deployments/deploy-poll.test.ts | Adds unit coverage for transient retries, terminal errors, deployment failures, persistent outages, and recovery after an earlier transient error. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Wait for polling interval] --> B{Self-hosted backend?}
B -->|Yes| C[Request deployment sync]
B -->|No| D[Read deployment status]
C --> D
D --> E{Request succeeded?}
E -->|No| F{Transient error?}
F -->|Yes| A
F -->|No| G[Fail immediately]
E -->|Yes| H{Deployment status}
H -->|READY| I[Return live deployment]
H -->|ERROR or CANCELED| G
H -->|Still building| J{Polling window expired?}
J -->|No| A
J -->|Yes| K[Return timeout result]
Reviews (4): Last reviewed commit: "fix(deployments): treat 429/408 as trans..." | Re-trigger Greptile
jwfing
left a comment
There was a problem hiding this comment.
Summary
Focused, well-tested fix that makes pollDeployment() tolerate transient gateway 5xx responses on the deployment status/sync endpoints while still failing fast on real terminal errors.
Requirements context
No matching spec/plan found under docs/specs/ (the repo's spec dir holds only diagnose-command and db-migrations-command designs; there is no docs/superpowers/). Assessed against the PR description and the referenced user feedback (81b47ca8) alone.
Findings
Critical
(none)
Suggestion
- Functionality —
src/commands/deployments/deploy.ts:317. The new guard rethrows anyCLIErrorwithstatusCode < 500, which includes429 Too Many Requests. A 429 from the gateway mid-poll is conventionally transient and does not indicate the deployment itself failed — the same reasoning the PR applies to 5xx. If the status endpoint is ever rate-limited during a long poll, this would abort the command even though the deployment is still progressing. Consider treating429(and possibly408) as transient alongside>= 500. Low blast radius, so non-blocking.
Information
- Software engineering —
src/commands/deployments/deploy-poll.test.ts. The three tests cover the fix's branches cleanly (5xx skip-to-READY, 4xx fail-fast, deploymentERROR), and the fake-timer usage is correct (each iteration awaits the interval before fetching, soadvanceTimersByTimeAsync(POLL_INTERVAL_MS * 3)maps to exactly three polls). One uncovered path worth a follow-up test: a persistent 5xx that never recovers should fall through to the timeout and returnisReady: false(surfacing the "did not finish within 5 minutes" message atdeploy.ts:527-539) rather than throwing — this is the user-visible behavior change and currently has no direct assertion. Not required to merge. - The
pollDeploymentexport, the mock path (../../lib/api/oss.jsmatches the import indeploy.ts:8), and thevi.hoisted+vi.mockpattern are all consistent with existing repo conventions (metadata-slug.test.ts,collectDeploymentFiles/createZipBufferexports).
Dimensions with no findings
- Security — no security-relevant changes; no new user input reaches SQL/shell/HTTP, no secrets/PII newly logged, auth is untouched, and no dependencies were added.
- Performance — no new queries, loops, or blocking work; poll interval/timeout are unchanged. The change only narrows which errors are rethrown.
Verdict
approved (informational — human approval via the separate approve flow). The fix is correct, in-scope, and matches its tests; the 429 note and the persistent-5xx test are optional improvements the author may take or leave.
…nt set
Follow-up to the transient-5xx fix. Three gaps addressed:
1. A 5xx that persisted for the whole 5-minute poll window was swallowed
silently: pollDeployment returned isReady=false and the caller printed
"Deployment is still building / did not finish within 5 minutes" with exit
0, never mentioning that all 60 status reads had failed. A real backend
outage was indistinguishable from a slow build. The loop now remembers the
most recent transient failure and returns it as `lastError` (cleared as soon
as a read succeeds, so a slow build is still reported as a plain timeout);
both not-ready output paths print it, and `deployments deploy --json`
includes it.
2. The spinner's elapsed counter only advanced on a successful read, so a run
of failed reads looked like a hung command. It now updates on the retry path
too.
3. `statusCode >= 500` was both too wide and too narrow. 501 means the route
does not exist on this backend and will not start working mid-poll, so it
now fails fast; 408/429 are genuinely retryable and polling every 5s is
exactly what trips a rate limit, so they are now tolerated. Transient set is
an explicit {408, 429, 500, 502, 503, 504}.
Also: create.ts claimed "did not finish within 2 minutes" while the timeout is
5 minutes — both messages now derive from POLL_TIMEOUT_MS so they cannot drift.
Tests: 7 new cases (persistent 5xx reports lastError, persistent network error,
lastError cleared after a successful read, 429 tolerated, 501 fails fast,
statusCode-less CLIError fails fast, transient failure on the sync request with
syncBeforeRead). The test file now imports the poll constants instead of
re-declaring them. Suite: 646 passed / 13 skipped.
A local symlink used to run the test suite from a worktree; .gitignore lists node_modules/ with a trailing slash, which matches a directory but not a symlink, so git add -A picked it up. Nothing else in the tree changes.
jwfing
left a comment
There was a problem hiding this comment.
Review: fix(deployments): tolerate transient gateway 5xx during deploy status polling
Summary: A well-scoped, thoroughly-tested fix that stops a single transient gateway 5xx (or 408/429/network error) from aborting deployments deploy while the deployment itself keeps running server-side, and surfaces lastError so a full-window outage is distinguishable from a slow build.
Requirements context
No matching spec/plan found under docs/specs/ (existing specs cover the diagnose and db-migrations commands only; there is no docs/superpowers/). Assessed against the PR description and the referenced user feedback 81b47ca8 ("deployments deploy/status intermittently fail with 'OSS request failed: 502' while the deployment succeeds"). The change directly addresses that intent.
Findings
Critical
(none)
Suggestion
- Functionality — app-layer
500is retried for the entire poll window (src/commands/deployments/deploy.ts:30,:334-339). Including500inTRANSIENT_POLL_STATUSESmeans a backend whose status route hard-500s (a persistent app bug, not a gateway blip) keeps the CLI spinning for the full 5 minutes rather than failing fast like501does. This is a reasonable, deliberately-documented trade-off and is now mitigated by reportinglastErroron timeout — flagging only so the choice is conscious. No change required.
Information
- Software engineering — excellent test coverage.
deploy-poll.test.tsexercises transient 502→READY, 429 tolerance, transient sync-request failure, 404/501/no-statusCode/ERROR fail-fast, persistent-5xx and persistent-networklastErrorreporting, and the "slow build leaveslastErrornull" case. This matches the repo's TDD convention and the coverage is genuinely comprehensive. The terminal-vs-transient classification (src/commands/deployments/deploy.ts:334-336) correctly relies onossFetchattachingres.statusasCLIError.statusCode(src/lib/api/oss.ts:225) and on deployment ERROR/CANCELED CLIErrors carrying nostatusCode(deploy.ts:315). - Functionality — pre-existing wording bug fixed as a bonus.
create.tspreviously reported "did not finish within 2 minutes" while the real timeout is 5 min (POLL_TIMEOUT_MS = 300_000); both call sites now usePOLL_TIMEOUT_MINUTES(create.ts:496,deploy.ts:580). Good consolidation. - Functionality — minor label imprecision. When
syncBeforeReadis set and the sync POST (not the status GET) fails at the network layer,formatFetchError(err, 'the deployment status endpoint')(deploy.ts:344) attributes it to the status endpoint. Harmless for user messaging; noting only for accuracy. - Security: no security-relevant changes — no new user input reaches SQL/shell/HTTP, no secrets logged (
lastErrorcarries only HTTP status / network-error text), no auth/authz changes, no new dependencies. - Performance: no concerns — poll cadence (
POLL_INTERVAL_MS) and window (POLL_TIMEOUT_MS) are unchanged; the added work per iteration is a string assignment. Retrying500for the whole window is bounded by the existing 5-minute timeout.
Scope note: the standalone deployments status command (status.ts) is a one-shot read, not a poller, so the "deploy/status" in the feedback refers to the status request inside the deploy loop — correctly the only site touched. The accidental node_modules symlink from an intermediate commit was cleaned up in the final HEAD (45aec86).
Verdict
approved (informational — a human still gives the explicit GitHub approval). Zero Critical findings; the implementation solves the reported problem, is well-tested, in-scope, and free of security/performance regressions. The Suggestion and Information items are non-blocking.
The previous follow-up went well beyond the one issue that warranted a pre-merge change. Reverted the scope creep, keeping only the actual defect: A 5xx persisting for the entire 5-minute poll window was swallowed silently — pollDeployment returned isReady=false and the caller printed "Deployment is still building / did not finish within 5 minutes" with exit 0, never mentioning that every status read had failed, so a backend outage was indistinguishable from a slow build. The loop now remembers the most recent transient failure and returns it as `lastError`, cleared as soon as a read succeeds so a genuinely slow build is still reported as a plain timeout. Both output paths surface it (`deployments deploy --json` includes it). Reverted from the previous commits: - TRANSIENT_POLL_STATUSES / 501 fail-fast / 408+429 tolerance — a follow-up- grade tuning change, explicitly flagged as non-blocking in review. Restored the original `statusCode === undefined || statusCode < 500` condition. - spinner update on the retry path — same, follow-up grade. - create.ts entirely, including the "2 minutes" wording bug and POLL_TIMEOUT_MINUTES. A real bug, but unrelated to this PR and dragged in. - The four tests covering the reverted behaviour (429, 501, no-statusCode, syncBeforeRead). Net vs the original PR: deploy.ts +31/-7 and 3 new tests, touching the same two files the PR already touched. Suite: 642 passed / 13 skipped.
Applies the one Suggestion from the initial review of this PR: the guard rethrew any CLIError with statusCode < 500, which includes 429 Too Many Requests. A rate limit on the status endpoint mid-poll says nothing about the deployment, which keeps running server-side — and polling every 5s is exactly the shape that trips a rate limit. 408 Request Timeout is retryable for the same reason. Implemented as an exception list checked alongside the existing >= 500 rule rather than a rewrite: the >= 500 condition is untouched, so 5xx (including 501) keeps its current behaviour. Only 408 and 429 change from terminal to transient; every other 4xx still fails fast. Test: it.each([408, 429]) asserting the poll continues to READY and lastError stays null. Suite: 644 passed / 13 skipped.
jwfing
left a comment
There was a problem hiding this comment.
Summary
Focused, well-tested bug fix that stops a single transient gateway 5xx (and 408/429/network read error) on the deployment status endpoint from aborting an otherwise-successful insforge deployments deploy; the diff is in scope and the logic is correct.
Requirements context
No matching spec/plan found — docs/specs/ only holds the diagnose and db-migrations designs, and there is no docs/superpowers/ directory. Assessed against the PR description, the linked feedback item (81b47ca8 — deploy/status intermittently failing with OSS request failed: 502 while the deploy actually succeeds), and surrounding code.
I verified the core assumption the fix rests on: ossFetch throws new CLIError(message, 1, err.error, res.status) (src/lib/api/oss.ts:230), so HTTP failures carry the numeric HTTP status in statusCode, and genuine network failures surface as a non-CLIError TypeError. The terminal/transient classification in pollDeployment keys off exactly these, so it behaves as intended. I also ran the new test file locally: 8/8 pass.
Findings
Critical
(none)
Suggestion
- Functionality — sibling
createflow doesn't surface the newlastError(src/commands/create.ts:484-498). Theinsforge createcommand also deploys viadeployProject()and has its own "still building" branch, but it neither readsresult.lastErrornor was it touched here (the PR description notes this follow-up was intentionally reverted to keep scope narrow). That's a reasonable call, but it means the "status endpoint unreachable vs. slow build" distinction this PR adds is only visible fromdeployments deploy, not fromcreate. Worth a fast-follow for consistency. Separately, that branch's message still says "Deployment did not finish within 2 minutes" whilePOLL_TIMEOUT_MSis 300_000 (5 min) — pre-existing and out of this PR's scope, flagging only so it isn't lost.
Information
- Software engineering —
syncBeforeRead: truetransient path is untested (src/commands/deployments/deploy.ts:299-301). All new tests callpollDeployment(..., false). A 5xx thrown by thePOST /syncrequest (legacy/self-hosted path) hits the same catch and is now treated as transient too — correct, but uncovered. A one-line case would lock it in. - Behavior change worth noting for reviewers — persistent outage now consumes the full 5-minute window. Because 5xx/408/429/network errors no longer fail fast, a status endpoint that is down for the entire poll window will now poll until
POLL_TIMEOUT_MSbefore returning (withlastErrorpopulated) rather than erroring immediately. This is the intended trade-off (the deployment may still be succeeding server-side) and the window is bounded, so it's not a performance concern — just a deliberate latency shift on the sad path.
Nice touches: resetting lastTransientError to null after any successful read (deploy.ts:317-319) so a slow-but-healthy build isn't mislabeled an outage, the explicit TRANSIENT_4XX_STATUSES set with a rationale comment, and the "final read succeeded → lastError null" test guarding exactly that reset. Test coverage across the transient/terminal/timeout matrix is thorough. No security-relevant changes (no new user input reaching SQL/shell/HTTP; lastError carries only backend-controlled error strings and host/error-code diagnostics, no secrets).
Verdict
approved — no Critical findings; the Suggestion/Information items are non-blocking. (Informational verdict for the bot's report; a human still gives the explicit GitHub approval.)
What was broken
insforge deployments deploypolls the deployment status endpoint every 5s until READY.pollDeployment()'s catch rethrew everyCLIError, including HTTP-level errors surfaced byossFetch— so a single transient gateway 502 on the status (or sync) request mid-poll aborted the command withError: OSS request failed: 502even though the deployment itself continued and reached READY on Vercel.What changed
In
src/commands/deployments/deploy.tspollDeployment(), the catch now only rethrowsCLIErrors that are terminal:statusCode), andstatusCode < 500).CLIErrors withstatusCode >= 500are treated as transient — same as network-level fetch errors already were — and polling continues.pollDeploymentis now exported for unit testing (matchingcollectDeploymentFiles/createZipBufferin the same file).Verification
src/commands/deployments/deploy-poll.test.ts(vitest, fake timers, mockedossFetch):ERRORstatus still fails.npm run lint→ 639 tests passed / 13 skipped, eslint clean apart from a pre-existing unrelated warning insrc/lib/api/apify.test.ts.No agent-skills update needed: no flags, defaults, or output shape changed.
Addresses user feedback 81b47ca8-b7ab-4f01-89be-5ca899b37102 (cli): deployments deploy/status intermittently fail with 'OSS request failed: 502' while deployment succee
🤖 Generated with Claude Code
Summary by cubic
Tolerate transient 5xx/server, 408/429, and network read errors during deployment status polling, and surface why polling timed out. Fixes intermittent “OSS request failed: 502” and makes timeouts clearer (feedback 81b47ca8).
pollDeployment(), fail fast only for deployment-thrownCLIErrors (nostatusCode) and non-retryable 4xx; treat 5xx, 408/429, and network failures as transient and keep polling. Clear any transient error after a successful read.lastError; shown in CLI and--jsonto distinguish “slow build” from “status endpoint unreachable.” ExportedpollDeployment(),POLL_INTERVAL_MS,POLL_TIMEOUT_MS. Addedvitesttests for transient 5xx/408/429/network tolerance, 4xx fail-fast, deploymentERROR, persistent outage reporting, andlastErrorclearing.Written for commit 262e018. Summary will update on new commits.
Note
Tolerate transient 5xx errors during deployment status polling in
pollDeploymentPreviously, any
CLIErrorthrown during polling would terminate the poll loop immediately. Now,pollDeploymentin deploy.ts only terminates early for locally-thrown deployment failures (nostatusCode) or client-side 4xx errors; 5xx gateway/server errors and network-level fetch errors are treated as transient and polling continues. Tests covering these three cases are added in deploy-poll.test.ts.Changes since #212 opened
pollDeploymentfunction to retry transient HTTP errors and track last error [2d60b54]createanddeploycommand handlers to report dynamic timeout duration and last polling error on timeout [2d60b54]node_modulesentry [45aec86]pollDeploymentto treat 5xx HTTP status codes and network-level fetch failures as transient errors that continue polling, while treating 4xx status codes andCLIErrorinstances withoutstatusCodeas terminal errors that abort immediately [94f3a41]POLL_TIMEOUT_MINUTESconstant and hardcoded timeout warning messages to fixed strings in deployment commands [94f3a41]pollDeploymenttest suite [94f3a41]pollDeploymentfunction to retry on transient 4xx HTTP status codes and added corresponding test coverage [262e018]Macroscope summarized e878664.