Skip to content

Add new recipes for Intel XPU(B60, B70) - #753

Open
Dboyqiao wants to merge 6 commits into
vllm-project:mainfrom
Dboyqiao:recipes_for_xpu
Open

Add new recipes for Intel XPU(B60, B70) #753
Dboyqiao wants to merge 6 commits into
vllm-project:mainfrom
Dboyqiao:recipes_for_xpu

Conversation

@Dboyqiao

@Dboyqiao Dboyqiao commented Aug 6, 2026

Copy link
Copy Markdown

for nemotron-3-nano, gemma-4 and qwen3.5/qwen3.6

@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
vllm-recipes Ready Ready Preview Aug 7, 2026 5:22am

Request Review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support and deployment documentation for Intel Arc Pro B60 and B70 (Intel XPU) hardware across various model recipes, including Gemma 4, Qwen 3.5, Qwen 3.6, and NVIDIA Nemotron 3. The review feedback points out critical configuration issues that would lead to out-of-memory (OOM) errors. Specifically, the NVIDIA Nemotron 3 Nano 30B recipe incorrectly configures a single-GPU setup with a tensor parallel size of 1 instead of a multi-GPU setup with a tensor parallel size of 4. Additionally, several Qwen model configurations are missing essential hardware_overrides for --tensor-parallel-size and --max-model-len to align with their validated Docker run commands.

Comment thread models/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16.yaml Outdated
Comment thread models/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16.yaml Outdated
Comment thread models/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16.yaml Outdated
Comment on lines +83 to +86
hardware_overrides:
xpu:
extra_args:
- "--enforce-eager"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The hardware_overrides.xpu configuration is missing the --tensor-parallel-size and --max-model-len parameters. Without these, running the recipe on XPU will default to TP=1 and full context length, which will cause an immediate OOM. Please add these overrides to match the validated Docker command.

hardware_overrides:
  xpu:
    extra_args:
      - "--enforce-eager"
      - "--tensor-parallel-size"
      - "4"
      - "--max-model-len"
      - "8192"

Comment on lines +87 to +90
hardware_overrides:
xpu:
extra_args:
- "--enforce-eager"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The hardware_overrides.xpu configuration is missing the --tensor-parallel-size and --max-model-len parameters. Without these, running the recipe on XPU will default to TP=1 and full context length, which will cause an immediate OOM. Please add these overrides to match the validated Docker command.

hardware_overrides:
  xpu:
    extra_args:
      - "--enforce-eager"
      - "--tensor-parallel-size"
      - "4"
      - "--max-model-len"
      - "8192"

Comment on lines +135 to +138
hardware_overrides:
xpu:
extra_args:
- "--enforce-eager"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The hardware_overrides.xpu configuration is missing the --tensor-parallel-size and --max-model-len parameters. Without these, running the recipe on XPU will default to TP=1 and full context length, which will cause an immediate OOM. Please add these overrides to match the validated Docker command.

hardware_overrides:
  xpu:
    extra_args:
      - "--enforce-eager"
      - "--tensor-parallel-size"
      - "4"
      - "--max-model-len"
      - "8192"

Comment on lines +138 to +140
xpu:
extra_args:
- "--enforce-eager"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The hardware_overrides.xpu configuration is missing the --tensor-parallel-size and --max-model-len parameters. Without these, running the recipe on XPU will default to TP=1 and full context length, which will cause an immediate OOM. Please add these overrides to match the validated Docker command.

  xpu:
    extra_args:
      - "--enforce-eager"
      - "--tensor-parallel-size"
      - "4"
      - "--max-model-len"
      - "8192"

Comment on lines +67 to +70
hardware_overrides:
xpu:
extra_args:
- "--enforce-eager"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The hardware_overrides.xpu configuration is missing the --max-model-len parameter. Without this, running the recipe on XPU will default to the full context length, which will cause an OOM. Please add this override to match the validated Docker command.

hardware_overrides:
  xpu:
    extra_args:
      - "--enforce-eager"
      - "--max-model-len"
      - "8192"

@Dboyqiao

Dboyqiao commented Aug 6, 2026

Copy link
Copy Markdown
Author

@jikunshang Could you help to review?

Comment thread models/Google/gemma-4-12B-it.yaml Outdated
VLLM_WORKER_MULTIPROC_METHOD: "spawn"
extra_args:
- "--enforce-eager"
- "--attention-backend"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's default option, we can remove.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept --attention-backend flash_attn intentionally. For Gemma 4 on XPU, if the attention backend is not explicitly set, vLLM treats Gemma 4 as a multimodal prefix-LM model and the XPU backend selector falls back to Triton attention because Flash Attention does not support the multimodal prefix-LM mask path.
Removed other args as per suggested.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it thanks.

Comment thread models/Google/gemma-4-12B-it.yaml Outdated
@jikunshang

Copy link
Copy Markdown
Member

please fix dco issue.

yintong-lu and others added 2 commits August 7, 2026 17:30
Signed-off-by: Yintong Lu <yintong.lu@intel.com>
Signed-off-by: Zhefeng, Qiao <zhefeng.qiao@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants