Skip to content

adjust CUDA_VISIBLE_DEVICES for TP > 1 - #1

Open
ptarasiewiczNV wants to merge 11 commits into
NickLucche:heterogenous-tpfrom
ptarasiewiczNV:fix-cuda-visible-devices
Open

adjust CUDA_VISIBLE_DEVICES for TP > 1#1
ptarasiewiczNV wants to merge 11 commits into
NickLucche:heterogenous-tpfrom
ptarasiewiczNV:fix-cuda-visible-devices

Conversation

@ptarasiewiczNV

@ptarasiewiczNV ptarasiewiczNV commented May 16, 2025 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown

When TP > 1 CUDA_VISIBLE_DEVICES should include all required devices. Also adjust decoder side channel ports for TP > 1.

NickLucche and others added 10 commits May 13, 2025 07:38
@NickLucche
split kv_cache along head dim
afccc5f 
fix descr indexing

change remote worker selection indexing; test ptp2-dtp4

Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
rebase cruft
e96f832 
Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
address race condition
f0ba728 
Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
integration test
d0c44bb 
Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
address review: move src_xfer_side_handle init out of hotpath and docs
bde92a5 
Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
address review: clarity
f674fc7 
Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
typing
230a8b8 
Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
revert proxy server
8274635 
Signed-off-by: nicklucche <nlucches@redhat.com>
@NickLucche
fix req_finished sync bug
abc710a 
Signed-off-by: nicklucche <nlucches@redhat.com>
@ptarasiewiczNV
adjust CUDA_VISIBLE_DEVICES for TP > 1
d5d3184
@github-actions

github-actions Bot commented May 16, 2025

Copy link
Copy Markdown

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ptarasiewiczNV @NickLucche