Add ThaiLLM/ThaiLLM-30B recipe (first Thai/SEA model, DGX Spark verified NVFP4) - #640
Add ThaiLLM/ThaiLLM-30B recipe (first Thai/SEA model, DGX Spark verified NVFP4)#640spped2000 wants to merge 2 commits into
Conversation
First Thai/SEA-language model recipe: ThaiLLM-30B (Qwen3-30B-A3B MoE continued-pretrain, Thailand's national open-model initiative) with a community NVFP4 variant verified end-to-end on DGX Spark (GB10), backed by a paired BF16-vs-NVFP4 Thai/English evaluation. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: spped2000 <spped2000@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Code Review
This pull request adds the configuration and deployment guide for the ThaiLLM-30B model, as well as registering ThaiLLM as a provider. The review feedback highlights that the minimum VRAM for the NVFP4 variant should be increased from 19 GB to 24 GB to provide sufficient headroom for vLLM initialization and prevent Out-Of-Memory errors.
| nvfp4: | ||
| model_id: "NatdhanaiPython/ThaiLLM-30B-NVFP4" | ||
| precision: nvfp4 | ||
| vram_minimum_gb: 19 |
There was a problem hiding this comment.
The vram_minimum_gb for the nvfp4 variant is set to 19. Since the quantized weights alone are 18.1 GB, this leaves only 0.9 GB of headroom for the CUDA context, activation memory, and KV cache. vLLM typically requires at least 2–4 GB of overhead just to initialize, meaning a 19 GB limit will almost certainly lead to Out-Of-Memory (OOM) errors during startup or early generation. Considering that the next standard GPU memory tier is 24 GB (e.g., RTX 3090/4090, L4), setting this to 24 is a much safer and more realistic minimum.
vram_minimum_gb: 24|
Thanks for the review. The 19 GB value follows the CONTRIBUTING VRAM formula (params x bytes_per_param x 1.2 = 31 x 0.5 x 1.2 = 18.6 -> 19), the same convention as the merged Qwen3.6-35B-A3B recipe's NVFP4 variant (21 GB = 35 x 0.5 x 1.2). For reference, measured on DGX Spark this checkpoint uses 16.9 GiB for weights plus runtime overhead and serves comfortably. Happy to bump to 24 GB if maintainers prefer real-world headroom over the formula convention here. |
The NVFP4 checkpoint moved from NatdhanaiPython/ThaiLLM-30B-NVFP4 to AGIcafet/ThaiLLM-30B-NVFP4 (organization repo). HuggingFace redirects the old id, but point the recipe at the canonical location. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: spped2000 <spped2000@users.noreply.github.com>
Summary
Adds the first Thai (and first Southeast Asian) language-model recipe: ThaiLLM/ThaiLLM-30B, the 30B MoE model of Thailand's national open-model initiative (a Qwen3-30B-A3B continued-pretrain, ~63B tokens with a large Thai share -> serves through the standard
Qwen3MoeForCausalLMpath).dgx_spark_gb10: verified- both variants run end-to-end on a DGX Spark (GB10), NVFP4 measured at 2.3-2.5x BF16 decode (27 -> 63 tok/s single stream, 69 -> 175 at 4 streams), 61 GB -> 18.1 GB weightsThe NVFP4 variant is backed by a paired BF16-vs-NVFP4 evaluation (19,786 paired MC questions, McNemar exact tests; letter-scored ThaiExam 0.619 -> 0.614, not significant p=0.79; Thai byte-perplexity 0.268 -> 0.282 bits/byte). Methodology and reproduction scripts: https://github.com/spped2000/thaillm-nvfp4-dgx-spark - live report: https://thaillm.agicafet.com
Changes
models/ThaiLLM/ThaiLLM-30B.yaml(new recipe)src/lib/providers.js(new ThaiLLM provider; logo fetches cleanly viascripts/fetch-provider-logos.mjs)Validation
node scripts/build-recipes-api.mjs->✓ JSON API: 149 models (127 with recommended_command, 656 default-hw alternatives, 1167 per-hw renderings), 128 promoted variants, 8 strategies, 2 kv-store deployments, 2 platforms (1 variant collision skipped)- the collision warning pre-exists on a clean tree.AI assistance disclosure
Prepared with assistance from Claude Code (recipe authoring and validation); all serve configurations and benchmark numbers come from our own measured runs on DGX Spark, and the contributor takes full responsibility for the content. Commit carries a
Co-authored-by: Claudetrailer per the vLLM contributing guidelines.