fix: smoother progress for xet downloads

[tobocop2]

Fixes #4058.

Problem

Xet downloads barely report progress. For a 2.5GB file the bar updates about 9 times total -- 0% → 1.7% → 5.6% → 11% → 25% → 84% → 100%. That 25% to 84% gap is ~1.5GB of nothing. Looks completely stuck.

The cause: xet_get() and download_bucket_files() use a 1-arg callback. xet-core supports a 2-arg signature that gives fine-grained network-level updates instead.

Fix

make_xet_progress_callback(progress_bar, file_size) returns a 2-arg callback so xet-core gives us useful progress. Transfer bytes are scaled to file size so the bar tracks 0-100% correctly (xet dedup can make transfer size differ from file size).

Wired into both xet_get() and download_bucket_files(). No public API changes.

Demo

Qwen3 4B (2.5 GB, xet-stored) with a custom tqdm_class callback. Builds on #4056 which fixes custom tqdm_class in non-TTY environments.

Before (bar appears stuck)

After (smooth progress)

Note

Both downloads finish in roughly the same time (~80s). The difference is purely in how progress is reported.

Multi-file: snapshot_download Qwen3-8B (5 xet shards, 16.4 GB)

Before (bar barely moves)

After (smooth per-file progress)

Related

• feat: Add download progress callback & fix tqdm_class silenced in non-TTY #4051 / fix: add progress_updater param for granular download progress #4043 (earlier PRs that identified the granularity problem)
• Fix custom tqdm_class silently broken in non-TTY environments #4056 (fixes custom tqdm_class handling, orthogonal to this)

---

Note

Low Risk
Low risk: changes are limited to progress reporting callbacks for Xet downloads, with new tests covering scaling/capping and edge cases. Main risk is minor regressions in progress bar updates or callback compatibility with hf_xet.download_files.

Overview
Improves progress reporting for Xet-powered downloads.

Adds make_xet_progress_callback to provide the 2-argument progress callback expected by xet-core for fine-grained network updates, while scaling transfer bytes to the expected file size (and capping at 100%) so tqdm bars advance smoothly and finish correctly.

Wires the new callback into both xet_get() and HfApi.download_bucket_files(), and adds focused tests validating the callback signature, scaling behavior (including dedup/transfer-total mismatch), and edge cases like zero increments and unknown file sizes.

^{Reviewed by Cursor Bugbot for commit 6a8f8a3. Bugbot is set up for automated code reviews on this repo. Configure here.}