adjust CUDA_VISIBLE_DEVICES for TP > 1 - #1

Open

ptarasiewiczNV wants to merge 11 commits into

NickLucche:heterogenous-tpfrom

ptarasiewiczNV:fix-cuda-visible-devices

Commits on May 13, 2025

split kv_cache along head dim
NickLucche
committed
afccc5f
rebase cruft
NickLucche
committed
e96f832
address race condition
NickLucche
committed
f0ba728
integration test
NickLucche
committed
d0c44bb
address review: move src_xfer_side_handle init out of hotpath and docs
NickLucche
committed
bde92a5
address review: clarity
NickLucche
committed
f674fc7
typing
NickLucche
committed
230a8b8
revert proxy server
NickLucche
committed
8274635

Commits on May 14, 2025

fix req_finished sync bug
NickLucche
committed
abc710a

Commits on May 16, 2025

adjust CUDA_VISIBLE_DEVICES for TP > 1
ptarasiewiczNV
committed
d5d3184
fix prefill base gpu id
ptarasiewiczNV
committed
0e9ea01