Skip to content

Commit 09cb26d

Browse files
committed
modify default max token num in trtllm_allreduce_fusion
1 parent b314228 commit 09cb26d

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

fastdeploy/model_executor/layers/flashinfer_comm_fusion.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ def flashinfer_allreduce_residual_rmsnorm(
115115
residual: paddle.Tensor,
116116
weight: paddle.Tensor,
117117
eps: float = 1e-6,
118-
max_token_num: int = 4096,
118+
max_token_num: int = 2048,
119119
use_oneshot: Optional[bool] = None,
120120
trigger_completion_at_end: bool = False,
121121
fp32_acc: bool = False,

0 commit comments

Comments
 (0)