[WIP] GB300 DSV4 TRT PR - #2638
Closed
tzulingk wants to merge 2 commits into
Closed
Conversation
…oint Add a DeepSeek-V4-Pro nvfp4-DSpark Dynamo-TRT-LLM AgentX config for R200 (Vera Rubin): the dsv4-fp4-r200-dynamo-trt-agentx master-config key, a 3p1d-dep8 / 1x-dep8-decode conc-1631 (b96) recipe, ctx_ep8/gen_ep8 EPLB tables, and a perf-changelog entry. The b96 point matches native trtllm-serve within ~1.6% tps/gpu with a tighter TTFT tail after fixing frontend tokenization (8 GB L1 cache + fastokens), request_plane=tcp, KV events off, orchestrator_placement=first_decode, and a decode host_cache offload tier (KV-deadlock fix). DRAFT: internal Rubin image/versions are TODO placeholders for a maintainer to fill in with the public R200 image + runner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
…rontier points Extend the R200 frontier to span low->high concurrency: 1p6d-dep4-tep4 conc-6 (interactivity end, 28 GPU, eplb0) and 2p1d-dep8 conc-1631 (density trim, 24 GPU). The 2p1d density point reaches 250k tps/gpu, within ~1.1% of native trtllm-serve (252.8k), and is deadlock-free via the decode host_cache offload tier (this config deadlocked 5x without it). Registered both in the dsv4-fp4-r200-dynamo-trt-agentx search-space + perf-changelog. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
tzulingk
changed the title
kedarpotdar-nv
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP GB300 TRT