Skip to content

ci: pin torch<2.15 for MPS workflow until pytorch/pytorch#202529 is fixed - #8955

Open
delock wants to merge 1 commit into
masterfrom
gma/mps-pin-torch-pre-2.15
Open

delock wants to merge 1 commit into
masterfrom
gma/mps-pin-torch-pre-2.15

Conversation

@delock

@delock delock commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

mps-torch-latest has failed on every run since 2026-10-28 (nightly master runs included, e.g. runs #231 and #238), always at the Install PyTorch step within ~50s. Root cause is upstream, not a DeepSpeed change:

  • torch 2.15.0 (released 2026-10-28) replaced the MPS device-family check with MTLGPUFamilyApple7 (pytorch/pytorch#197200).
  • The paravirtual GPU of GitHub-hosted macOS runners only reports MTLGPUFamilyMac2, so torch.backends.mps.is_available() returns False ("Skipping device Apple Paravirtual device that does not support Metal 2.0") and the workflow's assert fires.
  • Tracked upstream as pytorch/pytorch#202529. Real Apple Silicon is unaffected; only virtualized GPUs (i.e. our CI runners) hit this.

This PR pins torch<2.15 (last known-good series: 2.14.1) in the workflow with a comment referencing the upstream issue, restoring CI signal until the fix ships (patch release or 2.16). It also unblocks unrelated PRs that touch accelerator/** and currently inherit a red MPS check, e.g. #8933.

Verification

Dispatched the workflow on both refs as a controlled A/B (runs):

run ref result
#244 master (torch 2.15.0) Install PyTorch failed in ~40s
#243 gma/mps-pin-torch-pre-2.15 (torch 2.14.x) Install PyTorch and Install DeepSpeed passed; unit tests running (link in run history)

The only delta between the two is the pin, which closes the causal chain.

Notes

AI assistance was used to investigate the CI failure history and draft this change; the diff is a two-line workflow pin plus comment, reviewed and test-dispatched by the submitter.

torch 2.15.0 replaced the MPS device-family check with MTLGPUFamilyApple7,
which the paravirtual GPU of GitHub-hosted macOS runners does not support,
so torch.backends.mps.is_available() returns False and this workflow has
failed at the install step on every run since 2026-10-28. Pin to the last
known-good series and revisit when pytorch/pytorch#202529 is resolved.

Signed-off-by: Ma, Guokai <guokai.ma@intel.com>
@delock
delock requested a review from loadams as a code owner October 30, 2026 08:17
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-30T08:18:47.919899Z 6a66326 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant