Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ title: 'Ultra-High Interactivity on NVIDIA GPUs? TileRT on InferenceX'
subtitle: 'Can TileRT software on NVIDIA GPUs compete with Cerebras, Groq LPU, and SambaNova? Batch size 1, disaggregated engine, high-throughput prefill engine, high-interactivity decode engine'
date: '2026-08-10'
publishDate: '2026-08-10'
modifiedDate: '2026-08-18'
tags:
- benchmark
- gpu
Expand All @@ -13,6 +14,8 @@ tags:
- tilert
- vllm
- glm5
- agentx
- agentic
---

_Originally published on the [SemiAnalysis newsletter](https://newsletter.semianalysis.com/p/ultra-high-interactivity-on-nvidia) on August 10, 2026._
Expand Down Expand Up @@ -235,9 +238,9 @@ Work is being done to simplify this, especially as software development can be a

AI coding agents accelerate tuning within known templates, but novel transformations still require expert judgment. A monolithic persistent kernel also reduces the usefulness of conventional per-kernel profiler timelines, making automated feedback loops more difficult.

## Next steps with TileRT and InferenceX
## Next steps with TileRT

We are actively working on moving TileRT benchmarking from InferenceX's single-turn 8k/1k to our new agentic coding benchmark, which we call AgentX. This scenario replays real Claude Code and Codex traces with long-context, multi-turn requests, realistic subagent activity, and dynamic tool-use delays. Its median input length is 140k tokens, while the theoretical median cache-hit rate roofline reaches 99.2%.
AgentX, InferenceX's long-context, multi-turn coding scenario, is now live for supported models and serving stacks. It replays privacy-preserving workload shapes derived from real Claude Code and Codex sessions, with long-context, multi-turn requests, realistic subagent activity, and dynamic tool-use delays. Its median input length is 140k tokens, while the theoretical median cache-hit rate roofline reaches 99.2%. Extending TileRT benchmarking from its current single-turn 8k/1k workload to AgentX remains the next step for this serving stack.

<Figure
src="/images/ultra-high-interactivity-on-nvidia/agentx-benchmark-characteristics.png"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ title: '在 NVIDIA GPU 上实现超高交互性?TileRT 登陆 InferenceX'
subtitle: '运行在 NVIDIA GPU 上的 TileRT 软件能否与 Cerebras、Groq LPU、SambaNova 竞争?批大小为 1、分离式引擎、高吞吐量预填充引擎、高交互性解码引擎'
date: '2026-08-10'
publishDate: '2026-08-10'
modifiedDate: '2026-08-18'
tags:
- benchmark
- gpu
Expand All @@ -13,6 +14,8 @@ tags:
- tilert
- vllm
- glm5
- agentx
- agentic
---

_本文最初于 2026 年 8 月 10 日发布在 [SemiAnalysis 通讯](https://newsletter.semianalysis.com/p/ultra-high-interactivity-on-nvidia)。_
Expand Down Expand Up @@ -235,9 +238,9 @@ TileRT 继承了 ASIC 厂商最大的弱点。静态提前编译意味着模型

AI 编码智能体能在已知模板内加速调优,但新颖的变换仍需专家判断。此外,单体式持久化 kernel 也削弱了传统逐 kernel 性能剖析时间线的作用,使自动化反馈闭环更加困难。

## TileRT 与 InferenceX 的下一步
## TileRT 的下一步

我们正在积极推进,将 TileRT 的基准测试从 InferenceX 的单轮 8k/1k 场景扩展到我们新的智能体编码基准——AgentX。该场景回放真实的 Claude Code 与 Codex 轨迹,包含长上下文、多轮请求、真实的子智能体活动与动态工具调用延迟。其输入长度中位数为 140k tokens,理论缓存命中率中位数屋顶线可达 99.2%。
InferenceX 的长上下文多轮编码场景 AgentX 现已在具备对应数据的模型与服务栈上正式上线。它回放从真实 Claude Code 与 Codex 会话衍生出的隐私保护工作负载形态,包含长上下文、多轮请求、真实的 subagent 活动与动态工具调用延迟。其输入长度中位数为 140k tokens,理论缓存命中率中位数屋顶线可达 99.2%。对 TileRT 而言,下一步仍是将目前的单轮 8k/1k 基准测试扩展到 AgentX

<Figure
src="/images/ultra-high-interactivity-on-nvidia/agentx-benchmark-characteristics.png"
Expand Down
4 changes: 2 additions & 2 deletions packages/app/cypress/e2e/overview.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ const AGENTX_LABEL_ZH = '长上下文多轮真实智能体场景(AgentX)';
/** Shared by both locales: the scenario is named after its acronym. */
const AGENTX_SHORT = 'AgentX';

const PAGE_TITLE = 'Inference Cost per Million Tokens';
const PAGE_TITLE_ZH = '推理每百万 token 成本';
const PAGE_TITLE = 'Agentic Inference Costs';
const PAGE_TITLE_ZH = '智能体推理成本';
const SOURCE_NOTE = 'Source: InferenceX & SemiAnalysis Market July 2026 AI Cloud TCO Model';
const SOURCE_LINK_TEXT = 'SemiAnalysis Market July 2026 AI Cloud TCO Model';
const SOURCE_NOTE_ZH = '来源:InferenceX 与 SemiAnalysis Market July 2026 AI Cloud TCO Model';
Expand Down
6 changes: 4 additions & 2 deletions packages/app/cypress/e2e/zh-pages.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ describe('Chinese (/zh) pages', () => {
});

it('renders the Chinese landing content', () => {
cy.get('[data-testid="intro-section"]').should('contain.text', '智能体推理基准测试');
cy.contains('h2', '探索 InferenceX').should('exist');
cy.contains('快速对比').should('exist');
});
Expand Down Expand Up @@ -34,7 +35,7 @@ describe('Chinese (/zh) pages', () => {
});

it('footer renders in Chinese with zh-internal links', () => {
cy.get('[data-testid="footer-brand-description"]').should('contain.text', '开源推理基准测试');
cy.get('[data-testid="footer-brand-description"]').should('contain.text', '智能体推理');
cy.get('[data-testid="footer-link-supporters"]')
.should('contain.text', '支持者')
.and('have.attr', 'href', '/zh/quotes');
Expand Down Expand Up @@ -62,7 +63,8 @@ describe('Chinese (/zh) pages', () => {

it('renders the Chinese SEO intro above the chart', () => {
cy.get('[data-testid="zh-tab-intro"]').within(() => {
cy.contains('h1', 'AI 推理基准测试').should('exist');
cy.contains('h1', '智能体推理基准测试').should('exist');
cy.contains('长上下文、多轮').should('exist');
});
});

Expand Down
22 changes: 11 additions & 11 deletions packages/app/src/app/about/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,21 @@ const faqJsonLd = {
})),
};

const ABOUT_DESCRIPTION =
'InferenceX benchmarks agentic and fixed-sequence AI inference across accelerators and serving stacks. AgentX is its long-context, multi-turn coding scenario.';

export const metadata: Metadata = {
title: 'About',
description:
'InferenceX is an independent, vendor neutral, reproducible benchmark which continuously benchmarks inference software across a wide range of AI accelerators.',
description: ABOUT_DESCRIPTION,
alternates: enAlternates('/about'),
openGraph: {
title: 'About | InferenceX',
description:
'InferenceX is an independent, vendor neutral, reproducible benchmark which continuously benchmarks inference software across a wide range of AI accelerators.',
description: ABOUT_DESCRIPTION,
url: `${SITE_URL}/about`,
},
twitter: {
title: 'About | InferenceX',
description:
'InferenceX is an independent, vendor neutral, reproducible benchmark which continuously benchmarks inference software across a wide range of AI accelerators.',
description: ABOUT_DESCRIPTION,
},
};

Expand All @@ -47,8 +47,8 @@ export default function AboutPage() {
<section>
<Card>
<h2 className="text-lg font-semibold mb-2">
Open Source Continuous Inference Benchmark trusted by Operators of Trillion Dollar
GigaWatt Scale Token Factories
Open-Source Continuous Agentic Inference Benchmark Trusted by Operators of
Trillion-Dollar, GigaWatt-Scale Token Factories
</h2>
<p className="text-muted-foreground mb-2">
As the world progresses exponentially towards AGI, software development and model
Expand All @@ -58,9 +58,9 @@ export default function AboutPage() {
</p>
<p className="text-muted-foreground mb-2">
<strong>InferenceX&trade;</strong> (formerly InferenceMAX) is our independent, vendor
neutral, reproducible benchmark which addresses these issues by continuously
benchmarking inference software across a wide range of AI accelerators that are
actually available to the ML community.
neutral, reproducible benchmark. It measures both fixed-sequence serving and AgentX,
our long-context, multi-turn agentic coding workload, across AI accelerators and
serving stacks available to the ML community.
</p>
<p className="text-muted-foreground">
Our open data & insights are widely adopted by the ML community, capacity planning
Expand Down
7 changes: 4 additions & 3 deletions packages/app/src/app/blog/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ import { SITE_URL, SITE_NAME, AUTHOR_NAME } from '@semianalysisai/inferencex-con

export const metadata: Metadata = {
title: 'Articles',
description: `Technical articles from ${SITE_NAME} by ${AUTHOR_NAME} — AI inference benchmarking, chip performance analysis, and ML infrastructure insights.`,
description: `Technical articles from ${SITE_NAME} by ${AUTHOR_NAME} on agentic inference benchmarks, AgentX results, chip performance, and ML infrastructure.`,
alternates: enAlternates('/blog'),
openGraph: {
title: `Articles | ${SITE_NAME} by ${AUTHOR_NAME}`,
description: 'AI inference benchmarking insights and chip performance analysis.',
description: 'Articles on agentic inference benchmarks, AgentX results, and chip performance.',
url: `${SITE_URL}/blog`,
},
};
Expand Down Expand Up @@ -49,7 +49,8 @@ export default async function BlogPage({
<Card>
<h2 className="text-2xl lg:text-4xl font-bold tracking-tight">Articles</h2>
<p className="mt-3 text-base lg:text-lg text-muted-foreground">
Insights on AI inference benchmarking, chip performance, and ML infrastructure.
Articles on agentic inference, AgentX results, chip performance, and ML
infrastructure.
</p>
<p className="text-sm text-muted-foreground">
New to the terminology?{' '}
Expand Down
2 changes: 1 addition & 1 deletion packages/app/src/app/compare/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ import { bucketComparePairsByVendor, formatModelList } from '@/lib/compare-ssr';

export const dynamic = 'force-dynamic';

const DESCRIPTION = `InferenceX is the independent, open-source chip inference benchmark from SemiAnalysis, with verified, reproducible nightly results. ${SUPPORTERS_LINE} Compare latency, throughput & cost head-to-head across DeepSeek V4 Pro, DeepSeek R1, Kimi K2, MiniMax M3, GLM 5, Qwen 3.5 & more.`;
const DESCRIPTION = `InferenceX is the independent, open-source chip inference benchmark from SemiAnalysis, with verified, reproducible results updated as configurations change. ${SUPPORTERS_LINE} Compare latency, throughput & cost head-to-head across DeepSeek V4 Pro, DeepSeek R1, Kimi K2, MiniMax M3, GLM 5, Qwen 3.5 & more.`;

export const metadata: Metadata = {
title: 'Chip Comparisons',
Expand Down
7 changes: 6 additions & 1 deletion packages/app/src/app/layout.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ export const metadata: Metadata = {
},
description: DESCRIPTION,
keywords: [
'AgentX scenario',
'agentic inference benchmark',
'agentic coding benchmark',
'long context inference benchmark',
'multi-turn inference benchmark',
'AI inference benchmark',
'GPU benchmark',
'LLM benchmark',
Expand Down Expand Up @@ -160,7 +165,7 @@ const jsonLd = {
'@id': `${SITE_URL}/#application`,
name: SITE_NAME,
description:
'Open-source AI inference benchmark dashboard. Compare chip performance for LLM inference across NVIDIA GB200, H100, AMD MI355X, and more.',
'InferenceX is an open-source agentic inference benchmark dashboard. It compares the AgentX long-context, multi-turn coding scenario with fixed-sequence serving across NVIDIA, AMD, and other accelerators.',
url: SITE_URL,
applicationCategory: 'DeveloperApplication',
operatingSystem: 'Web',
Expand Down
2 changes: 1 addition & 1 deletion packages/app/src/app/llms-full.txt/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export async function GET() {
`# ${SITE_NAME} Articles — Full Content`,
`> By ${AUTHOR_NAME}`,
'',
`This file contains the full text of all articles from ${SITE_NAME} (${SITE_URL}/blog).`,
`This file contains the full text of all articles from ${SITE_NAME} (${SITE_URL}/blog), covering agentic inference benchmarks including AgentX results, fixed-sequence AI inference, chip performance, and ML infrastructure.`,
`It is intended for consumption by large language models and AI assistants.`,
'',
'---',
Expand Down
2 changes: 1 addition & 1 deletion packages/app/src/app/llms.txt/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ export async function GET() {
const lines = [
`# ${SITE_NAME} by ${AUTHOR_NAME}`,
'',
`> ${SITE_NAME} is the open-source AI inference benchmark dashboard. We compare chip performance for LLM inference across NVIDIA GB200, H100, AMD MI355X, and more.`,
`> ${SITE_NAME} is an open-source agentic inference benchmark dashboard. It compares the AgentX long-context, multi-turn coding scenario with fixed-sequence serving across NVIDIA, AMD, and other accelerators using public runs.`,
'',
`## Links`,
'',
Expand Down
2 changes: 1 addition & 1 deletion packages/app/src/app/manifest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ export default function manifest(): MetadataRoute.Manifest {
name: `${SITE_NAME} by ${AUTHOR_NAME}`,
short_name: SITE_NAME,
description:
'Open-source AI inference benchmark. Compare chip performance across NVIDIA, AMD, and more.',
'Open-source agentic inference benchmark. Compare the AgentX scenario and fixed-sequence serving performance across NVIDIA, AMD, and more.',
start_url: '/',
display: 'standalone',
background_color: '#09090b',
Expand Down
8 changes: 4 additions & 4 deletions packages/app/src/app/overview/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,21 @@ import { getOverviewPageData } from '@/lib/overview-data.server';
export const dynamic = 'force-dynamic';

const DESCRIPTION =
'Compare hyperscaler cost per million total tokens across MI355X, B200, B300, GB200 and GB300 using the scenario shown for each active model.';
'Compare hyperscaler cost per million total tokens across MI355X, B200, B300, GB200, and GB300 for the AgentX long-context, multi-turn coding scenario and fixed-sequence scenarios where data is available.';

export const metadata: Metadata = {
title: 'Inference Cost Overview',
title: 'Agentic Inference Costs',
description: DESCRIPTION,
alternates: enAlternates('/overview'),
openGraph: {
title: `Inference Cost Overview | ${SITE_NAME}`,
title: `Agentic Inference Costs | ${SITE_NAME}`,
description: DESCRIPTION,
url: `${SITE_URL}/overview`,
type: 'website',
},
twitter: {
card: 'summary_large_image',
title: `Inference Cost Overview | ${SITE_NAME}`,
title: `Agentic Inference Costs | ${SITE_NAME}`,
description: DESCRIPTION,
},
};
Expand Down
18 changes: 9 additions & 9 deletions packages/app/src/app/zh/about/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -22,22 +22,22 @@ const faqJsonLd = {
})),
};

const ABOUT_DESCRIPTION =
'InferenceX 对比各类加速器与服务栈的智能体推理和固定序列 AI 推理性能。AgentX 是其长上下文多轮编码场景。';

export const metadata: Metadata = {
title: '关于',
description:
'InferenceX 是一个独立、厂商中立、可复现的基准测试平台,持续测试各类 AI 加速器上的推理软件性能。',
description: ABOUT_DESCRIPTION,
alternates: zhAlternates('/about'),
openGraph: {
title: '关于 | InferenceX',
description:
'InferenceX 是一个独立、厂商中立、可复现的基准测试平台,持续测试各类 AI 加速器上的推理软件性能。',
description: ABOUT_DESCRIPTION,
url: `${SITE_URL}/zh/about`,
locale: ZH_OG_LOCALE,
},
twitter: {
title: '关于 | InferenceX',
description:
'InferenceX 是一个独立、厂商中立、可复现的基准测试平台,持续测试各类 AI 加速器上的推理软件性能。',
description: ABOUT_DESCRIPTION,
},
};

Expand All @@ -49,16 +49,16 @@ export default function AboutPageZh() {
<section>
<Card>
<h2 className="text-lg font-semibold mb-2">
开源持续推理基准测试——受万亿美元级吉瓦规模 Token 工厂运营者的信赖
开源持续智能体推理基准测试——受万亿美元级吉瓦规模 Token 工厂运营者的信赖
</h2>
<p className="text-muted-foreground mb-2">
随着世界以指数级速度迈向
AGI,软件开发和模型发布日新月异。现有基准测试因其静态性质而迅速过时,参与者往往提交专为基准测试定制的软件镜像,无法反映真实的线上推理性能。
</p>
<p className="text-muted-foreground mb-2">
<strong>InferenceX&trade;</strong>(原名
InferenceMAX)是我们独立、厂商中立、可复现的基准测试平台,通过持续测试实际可用于 ML
社区的各类 AI 加速器上的推理软件来解决这些问题
InferenceMAX)是我们独立、厂商中立、可复现的基准测试平台。它测试固定序列推理服务和
AgentX 长上下文多轮智能体编码工作负载,覆盖 ML 社区实际可用的各类 AI 加速器与服务栈
</p>
<p className="text-muted-foreground">
我们的开放数据与洞察已被 ML 社区广泛采用,包括万亿美元级 Token 工厂和 AI
Expand Down
6 changes: 3 additions & 3 deletions packages/app/src/app/zh/blog/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ import { SITE_URL, SITE_NAME, AUTHOR_NAME } from '@semianalysisai/inferencex-con

export const metadata: Metadata = {
title: '文章',
description: `${SITE_NAME} by ${AUTHOR_NAME} 的技术文章——AI 推理基准测试、Chip 性能分析与 ML 基础设施洞见。`,
description: `${SITE_NAME} by ${AUTHOR_NAME} 发布关于智能体推理基准测试、AgentX 结果、Chip 性能与 ML 基础设施的技术文章。`,
alternates: zhAlternates('/blog'),
openGraph: {
title: `文章 | ${SITE_NAME} by ${AUTHOR_NAME}`,
description: 'AI 推理基准测试洞见与 Chip 性能分析。',
description: '关于智能体推理基准测试、AgentX 结果与 Chip 性能的文章。',
url: `${SITE_URL}/zh/blog`,
locale: ZH_OG_LOCALE,
},
Expand Down Expand Up @@ -51,7 +51,7 @@ export default async function ZhBlogPage({
<Card>
<h2 className="text-2xl lg:text-4xl font-bold tracking-tight">文章</h2>
<p className="mt-3 text-base lg:text-lg text-muted-foreground">
关于 AI 推理基准测试、Chip 性能与 ML 基础设施的深度洞见
关于智能体推理、AgentX 结果、Chip 性能与 ML 基础设施的文章
</p>
<p className="text-sm text-muted-foreground">
不熟悉相关概念?{' '}
Expand Down
2 changes: 1 addition & 1 deletion packages/app/src/app/zh/compare/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import { ZH_OG_LOCALE, zhAlternates } from '@/lib/i18n';

export const dynamic = 'force-dynamic';

const DESCRIPTION = `InferenceX 是 SemiAnalysis 推出的独立开源 Chip 推理基准测试平台,提供经过验证的、可复现的每夜测试结果。${SUPPORTERS_LINE_ZH}横向对比 DeepSeek V4 Pro、DeepSeek R1、Kimi K2、MiniMax M3、GLM 5、Qwen 3.5 等模型的延迟、吞吐量与成本。`;
const DESCRIPTION = `InferenceX 是 SemiAnalysis 推出的独立开源 Chip 推理基准测试平台,提供经过验证、可复现并随配置变化更新的测试结果。${SUPPORTERS_LINE_ZH}横向对比 DeepSeek V4 Pro、DeepSeek R1、Kimi K2、MiniMax M3、GLM 5、Qwen 3.5 等模型的延迟、吞吐量与成本。`;

export const metadata: Metadata = {
title: 'Chip 对比',
Expand Down
Loading