Repository navigation

ZeRO-1/2: preserve gradients across request boundaries until step - #8949

Merged

stas00 merged 3 commits into

stas/stale-grads

Nov 2, 2026

Merged

ZeRO-1/2: preserve gradients across request boundaries until step#8949
stas00 merged 3 commits into
masterfrom
stas/stale-grads

Commits on Oct 29, 2026
- ZeRO-1/2: preserve gradients across request boundaries until step
  
  seanbell
  authored and
  stas00
  committed
  bfa3c3d
Commits on Oct 30, 2026
- address review
  stas00
  committed
  7e00e1a
- ZeRO-1/2: allocate CPU gradient accumulation lazily
  stas00
  committed
  e5d68d9