[NPU Roadmap, Updated to 2026-Q2] NPU support for Liger-Kernel

[zheliuyu]

2026 Q2 Roadmap

Thanks to the Liger-Kernel team for their strong support of our work in Q1 2026. ❤🧡💛💜🤎💚💙

We will keep contributing to the Liger-Kernel + NPU community in Q2. This roadmap details the future plans for NPU native support. Welcome to join in the discussion.

Integrated into training software

In order to promote Liger-Kernel, we need to integrate it into real production scenarios to verify its performance in actual training and inference tasks.

We selected four training frameworks: VeOmni, LLaMA-Factory, verl and ms-swift, which are widely used in real production scenarios. We will continue to demonstrate the effectiveness of using Liger-Kernel in these software applications.

Software	APIs	First PR
VeOmni	LigerRMSNorm	ByteDance-Seed/VeOmni#415
	liger_rotary_pos_emb
	LigerSwiGLUMLP
LLaMA-Factory	apply_liger_kernel_to_qwen3	hiyouga/LlamaFactory#10386
verl	-	DOING
ms-swift	-	DOING

Performance Enhancement: Part 2

Following this plan, the performance of some kernels still hasn't met the standard and will be continuously optimized in Q2. For example:

• [NPU]: optimize cross entropy kernel gradient computation #1178
• [NPU]Improvement performence for grpo_loss #1174
• [NPU] Add fused_linear_cross_entropy operator #1164
• optimize group_norm for ASCEND_NPU #1154
• [Optimize, NPU] Remove tl.where from _rms_norm_forward/backward_kernel_tiled() #1153

Skills

Due to the differences in programming logic between triton-ascend and triton, migrating kernels to NPU inevitably incurs a time cost for performance optimization.

To address this issue, we have developed a series of skills for AI programming tools. For example: https://gitcode.com/Ascend/agent-skills/blob/master/skills/triton-operator-performance-optim/SKILL.md

Next, we will translate them into English and demonstrate the effectiveness of using these skills.

Support for more NPU machine types

Similarly, this includes three aspects: enabling functionality, ensuring accuracy within the error tolerance, and performance optimization.

Current support:

• Atlas 900 A2 POD (64G), which can be used for benchmark tasks.
• Atlas 800I A2 (32G), used by CI machines. This model NPU machine type not be used for benchmark work.

Planned support:

• Atlas 800T A3 (64G)

Improvement of NPU CI work

Currently, the NPU CI status badge can only be displayed in the README.

We plan to find a way to mark the CI status badge in NPU-related PRs. This would simulate the state after native CI integration and achieve the effect of checking PR quality.

2026 Q1 Roadmap

Thanks very much to Liger-Kernel for accepting our first native support pr.
This roadmap details the future plans for NPU native support. Welcome to join in the discussion.

NPU Native Support

This shows how Liger-Kernel works on NPU.

• Add Ascend NPU device support. #955
• [RFC] Code Organization for New Backend Support #965
• feat(NPU): add UB Manager for auto tiling strategy management #987
• [NPU]: remove num_stages and num_warps from NPU Triton kernels #1111

Unit Test Coverage Improvement: Functionality & Precision

The accuracy of all kernels in Liger-Kernel must be within the acceptable tolerance. This task can be accomplished by checking the actual execution results of all test cases in the ./test folder.

We have done the following work to ensure that all test cases under ./test/transformers have passed.

List of each kernel's first PR:

• cross_entroy: [NPU]:Add support for the cross_entropy operator. #1148
• dyt: [NPU]:Added support for the dyt operator #1124
• embedding: [NPU]: Add NPU support for the embedding #1028
• fused_add_rms_norm: [NPU]: NPU-optimized fused_add_rms_norm kernel #1070
• fused_linear_cross_entroy: [NPU] Add fused_linear_cross_entropy operator #1164
• fused_linear_jsd: [NPU]:Add support for the fused_linear_jsd operator. #1151
• fused_neighborhood_attention: [NPU]support fused neighborhood attention for npu #1034
• geglu: [NPU]: refine geglu memory_multiplier based on UB analysis #996
• group_norm: [NPU] Add group norm support on NPU #1144
• grpo_loss: [NPU] Add support for the grpo_loss #1146
• jsd: [NPU]:Added support for the jsd operator #1134
• kl_div: [NPU] Add kl_div support on NPU #1118
• layer_norm: [NPU]:Added support for the layer_norm operator in npu #1113
• llama4_rope: [NPU] Add Llama4_rope support on NPU #1035
• poly_norm: [NPU]:Added support for the poly_norm operator #1114
• qwen2vl_mrope: [NPU]: Add NPU support for the mrope operator #992
• rms_norm: [NPU]: NPU-optimized rms_norm kernel #1099
• rope: [NPU]: Add NPU support for the mrope operator #992
• softmax: [NPU] Add softmax implementation #1087
• sparsemax: [NPU] Add NPU optimized sparsemax #1104
• swiglu: [NPU]: Add NPU support for the swiglu #995
• tvd: [NPU]: Add NPU support for the tvd operator #998

Additionally, the test cases under ./test/transformers can serve as the foundation for NPU CI to guard future pull requests.

• [NPU]RFC: Ascend CI Integration #1022

Due to certain policy restrictions, Liger-Kernel cannot be natively integrated into NPU CI. Therefore, we have marked the NPU CI status badge in the README.

• [NPU] Add Ascend NPU CI status in README #1131

We will look for a more convenient solution in the future.

Performance Enhancement: Part 1

Unit tests can be used to APIs functionality and precision. However, as third-party devices may not fully align in their usage patterns of Triton, a performance optimization process is required.

Regarding the evaluation criteria, we have designed it as follows: The speedup ratio of each kernel compared to huggingface/torch should be greater than 1.

The progress of this task can be tracked in this rfc issue.

• [RFC, NPU] NPU benchmark - as a baseline for follow-up tasks. #1159

Benchmark Enhancement

To enable benchmarks to run reliably across devices with different memory capacities (e.g., 32G and 64G NPUs), we collaborated with the community to redesign the benchmark framework. The key improvements include:

1. Device memory awareness: Introduced runtime memory probing (estimate_kernel_peak_memory) to automatically detect available device memory and determine safe execution parameters, ensuring benchmarks can run successfully on different device types without OOM.

2. Standardized benchmark dimensions: Defined two orthogonal benchmark dimensions — D1 (non-model dimension sweep, e.g., sequence length) and D2 (model config sweep across real-world architectures from MODEL_REGISTRY), providing a more comprehensive and structured view of kernel performance.

Phase 1 (Foundation) and Phase 2 (Model-config sweep) have been completed. Phase 3 (Rollout and visualization) is planned for future work.

• [Test]: Refactor benchmark_geglu with standardized model configs #1116
• [Benchmark]: Add --sweep-mode and --bt to benchmark CLI. #1163

Co-Author: @momochen

[Tcc0403]

Is there any repo with npu ci where contributors can test around?

[Ginray]

Is there any repo with npu ci where contributors can test around?

We’ve built NPU CI for LlamaFactory, swift and verl, and will verify liger kernel’s compatibility with these libraries in follow-up work. Additionally, we hope to set up NPU CI for the liger kernel library, and can provide NPU machines to support this effort. We’d like to connect with you to move this forward.

[momochen]

@Ginray hey there, we are excited to see community supports to NPU, aligned with our long term goal to support heterogenous accelerators. Let's setup a formal meeting sometime this week on making NPU CI available. @Tcc0403 let's find a mutually convenient time.

[zheliuyu]

Some new players have joined the game. ✨

• VeOmini
  • LigerRMSNorm @ji-huazhong
  • LigerSwiGLUMLP @jiaqiw09

[Tcc0403]

let's find a mutually convenient time.

@momochen Any time is fine. I can work around it.

[TianHao324]

I am working on matters related to liger_rotary_pos_emb.

[zheliuyu]

I am working on matters related to liger_rotary_pos_emb.

cooool! Let's get started.

[pillumina]

FYI:
I'll work on this one: LigerLayerNorm

[UserChen666]

FYI:
I'll work on this one: LigerSiLUMulFunction

[TianHao324]

FYI:
liger_rotary_pos_emb ：Done
I'll work on this one: liger_multimodal_rotary_pos_emb

[Tcc0403]

Is there a working group for NPU support? It would be great if we can see what folks are working on!

[jiaqiw09]

Is there a working group for NPU support? It would be great if we can see what folks are working on!

@Tcc0403
Would it be possible to create a dedicated channel in official Discord, so we can share technical details and coordinate more efficiently to help build the ecosystem?

[Tcc0403]

Would it be possible to create a dedicated channel in official Discord, so we can share technical details and coordinate more efficiently to help build the ecosystem?

We have a discord channel in GPU MODE server and slack channel for Liger Kernel. Both are official channels where we used to discuss things, but they aren't that active recently. If you mean a dedicated discrod server, we'll need to discuss internally how we can run it.

[jiaqiw09]

We have a discord channel in GPU MODE server and slack channel for Liger Kernel. Both are official channels where we used to discuss things, but they aren't that active recently. If you mean a dedicated discrod server, we'll need to discuss internally how we can run it.

Got it, thanks!

I was referring to adding a new channel under the existing "Computing Platforms" category (similar to #cuda or #rocm) in discord, rather than creating a new server or a new top-level category. Sorry for the ambiguous wording earlier.

For example, something like a #ascend or #npu channel would already work very well for us.

[Tcc0403]

Oh, gpu mode doesn't belong to liger kernel team, so that's not in our control. But gpu mode is really cool with oss stuffs, I believe it's more than welcome to have npu channel there. cc @msaroufim @S1ro1

Also, as you request, @momochen is going to create liger kernel discord server for better collaboration with oss community!