Retry each code sample once on failure to absorb transient backend errors#5182
Conversation
Every Test Code Samples failure since the sample-bug fixes in #5120 has been a transient error that cleared within seconds: gRPC INTERNAL on DeleteDataPipeline, DEADLINE_EXCEEDED on data-query, or brief test machine connection drops. Run each sample a second time after a 30s delay before counting it as failed, so only persistent failures fail the job. Sample credentials already flow through each step's env block, so the redundant inline variable prefixes are dropped. Tracking issue: #5119
✅ Deploy Preview for viam-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Validation summary (8 dispatched runs on 7/13, details on #5119): The retry works and is needed. Across three dispatches of this branch, the retry absorbed three would-be job failures — The TypeScript connect failures seen during validation are unrelated to this change. All three branch dispatches failed the 7 machine-dialing TS samples with Generated by Claude Code |

Problem
Every Test Code Samples failure since the sample-bug fixes in #5120 (7/1) has been a transient backend error, not a code bug — triage across the failing runs in #5119:
INTERNALonDeleteDataPipeline(pipeline-delete.py/.go,pipeline-list.pyteardown,pipeline-create.gocleanup) — hit 6 of 8 runs on 7/13DEADLINE_EXCEEDEDon the shareddata-querySQL query (once taking out all three language variants in the same run)These clear within seconds: in run 29241295626, the pipeline orphaned by a failed delete was successfully deleted by the next sample's setup cleanup 3 seconds later.
Fix
Run each failed sample a second time after a 30-second delay before counting it as failed, in all three language steps. Only persistent failures fail the job. Samples are already re-runnable by design (the suite runs weekly and on every push, and each sample cleans up prior state on entry), so a retry is equivalent to the next scheduled run.
Also drops the redundant inline
VAR="$VAR"prefixes on the run commands: each step'senv:block already exports a superset of those variables to every process in the step.Validation (8 dispatched runs on 7/13)
pipeline-delete.py,pipeline-list.py,pipeline-create.go— each failed once, passed 30s later), while two plain-main control dispatches failed outright on the sameINTERNALerror.What this does not fix
Failures that persist longer than the 30s retry window — the flapping TS WebRTC incident, extended test-machine outages, and the late-afternoon persistent
DeleteDataPipelineerrors — still fail the job. Those are backend/infra issues tracked on #5119.Tracking issue: #5119
🤖 Generated with Claude Code