Delta weight sync using Xet buckets

[AmineDiro]

What does this PR do?

• Sparse weight sync between trainer and vLLM is working — encodes only changed bf16 elements as sparse safetensors (indices + values), uploads to HF Storage Bucket
•  ~20-35 MB per delta vs 1.2 GB full model (Qwen3-0.6B), sparsity >99%

Training converges correctly on the immediate-EOS sanity check 👍

Still have some optimizations to implement. Both trainer and vLLM hold a CPU bf16 snapshot of the model

• I tried predicting changes from Adam state (m, v) to skip the snapshot based on bf16 ULP but recall was only ~30%, ground truth difference needed instead. So maybe I am doing something incorrectly
• vLLM: snapshot needed because load_weights only accepts full tensors, no API for sparse in-place updates in vllm for now but maybe I can contribute this feature to vllm directly

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline, Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

AI writing disclosure

We welcome the use of AI tools to help with contributions. For transparency and to help us improve our review process, please indicate the level of AI involvement in this PR.

• No AI usage: the PR was written entirely by a human.
• AI-assisted: some parts were suggested or improved by AI, but the PR was written and reviewed by a human.
• AI-generated: the PR was mostly or fully generated by an AI tool.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

[qgallouedec]

That looks good, I'm not not sure to understand the big picture. Why do we push the weights (the diffs actually) if it's only to download them. Shouldn't we just keep the diff locally?

[qgallouedec]

vLLM: snapshot needed because load_weights only accepts full tensors, no API for sparse in-place updates in vllm for now but maybe I can contribute this feature to vllm directly

Let's ask them directly

[AmineDiro]

That looks good, I'm not not sure to understand the big picture. Why do we push the weights (the diffs actually) if it's only to download them. Shouldn't we just keep the diff locally?

I think the goal of this technique is mainly for completely disggregating the inference and trainer servers. Like if the inference is running on some HF space you can still exchange weights. It also provides a good checkpoint mechanism for the weights because you have that for free (although I need to check the Xet atomicity semantics :? )