Skip to content

Add ThaiLLM/ThaiLLM-30B recipe (first Thai/SEA model, DGX Spark verified NVFP4) - #640

Open
spped2000 wants to merge 2 commits into
vllm-project:mainfrom
spped2000:add-thaillm-30b
Open

Add ThaiLLM/ThaiLLM-30B recipe (first Thai/SEA model, DGX Spark verified NVFP4)#640
spped2000 wants to merge 2 commits into
vllm-project:mainfrom
spped2000:add-thaillm-30b

Conversation

@spped2000

Copy link
Copy Markdown

Summary

Adds the first Thai (and first Southeast Asian) language-model recipe: ThaiLLM/ThaiLLM-30B, the 30B MoE model of Thailand's national open-model initiative (a Qwen3-30B-A3B continued-pretrain, ~63B tokens with a large Thai share -> serves through the standard Qwen3MoeForCausalLM path).

  • default: BF16 base checkpoint (61 GB weights)
  • nvfp4: community TensorRT Model Optimizer checkpoint (NatdhanaiPython/ThaiLLM-30B-NVFP4), calibrated on a 50/50 Thai/English mix
  • hardware: dgx_spark_gb10: verified - both variants run end-to-end on a DGX Spark (GB10), NVFP4 measured at 2.3-2.5x BF16 decode (27 -> 63 tok/s single stream, 69 -> 175 at 4 streams), 61 GB -> 18.1 GB weights

The NVFP4 variant is backed by a paired BF16-vs-NVFP4 evaluation (19,786 paired MC questions, McNemar exact tests; letter-scored ThaiExam 0.619 -> 0.614, not significant p=0.79; Thai byte-perplexity 0.268 -> 0.282 bits/byte). Methodology and reproduction scripts: https://github.com/spped2000/thaillm-nvfp4-dgx-spark - live report: https://thaillm.agicafet.com

Changes

  • models/ThaiLLM/ThaiLLM-30B.yaml (new recipe)
  • src/lib/providers.js (new ThaiLLM provider; logo fetches cleanly via scripts/fetch-provider-logos.mjs)

Validation

node scripts/build-recipes-api.mjs -> ✓ JSON API: 149 models (127 with recommended_command, 656 default-hw alternatives, 1167 per-hw renderings), 128 promoted variants, 8 strategies, 2 kv-store deployments, 2 platforms (1 variant collision skipped) - the collision warning pre-exists on a clean tree.

AI assistance disclosure

Prepared with assistance from Claude Code (recipe authoring and validation); all serve configurations and benchmark numbers come from our own measured runs on DGX Spark, and the contributor takes full responsibility for the content. Commit carries a Co-authored-by: Claude trailer per the vLLM contributing guidelines.

First Thai/SEA-language model recipe: ThaiLLM-30B (Qwen3-30B-A3B MoE
continued-pretrain, Thailand's national open-model initiative) with a
community NVFP4 variant verified end-to-end on DGX Spark (GB10),
backed by a paired BF16-vs-NVFP4 Thai/English evaluation.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: spped2000 <spped2000@users.noreply.github.com>
@vercel

vercel Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
vllm-recipes Ready Ready Preview, Comment Jul 21, 2026 9:23pm

Request Review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds the configuration and deployment guide for the ThaiLLM-30B model, as well as registering ThaiLLM as a provider. The review feedback highlights that the minimum VRAM for the NVFP4 variant should be increased from 19 GB to 24 GB to provide sufficient headroom for vLLM initialization and prevent Out-Of-Memory errors.

nvfp4:
model_id: "NatdhanaiPython/ThaiLLM-30B-NVFP4"
precision: nvfp4
vram_minimum_gb: 19

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The vram_minimum_gb for the nvfp4 variant is set to 19. Since the quantized weights alone are 18.1 GB, this leaves only 0.9 GB of headroom for the CUDA context, activation memory, and KV cache. vLLM typically requires at least 2–4 GB of overhead just to initialize, meaning a 19 GB limit will almost certainly lead to Out-Of-Memory (OOM) errors during startup or early generation. Considering that the next standard GPU memory tier is 24 GB (e.g., RTX 3090/4090, L4), setting this to 24 is a much safer and more realistic minimum.

    vram_minimum_gb: 24

@spped2000

Copy link
Copy Markdown
Author

Thanks for the review. The 19 GB value follows the CONTRIBUTING VRAM formula (params x bytes_per_param x 1.2 = 31 x 0.5 x 1.2 = 18.6 -> 19), the same convention as the merged Qwen3.6-35B-A3B recipe's NVFP4 variant (21 GB = 35 x 0.5 x 1.2). For reference, measured on DGX Spark this checkpoint uses 16.9 GiB for weights plus runtime overhead and serves comfortably. Happy to bump to 24 GB if maintainers prefer real-world headroom over the formula convention here.

The NVFP4 checkpoint moved from NatdhanaiPython/ThaiLLM-30B-NVFP4 to
AGIcafet/ThaiLLM-30B-NVFP4 (organization repo). HuggingFace redirects
the old id, but point the recipe at the canonical location.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: spped2000 <spped2000@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant