是否支持无实体卡时workload的生成

[oyrtH]

当前计算耗时仅支持通过aiob基于目标型号实体卡上的估算，未来是否支持无实体卡时workload的生成？以及是否支持指定网络结构与并行策略下的timeline估算？

[tianhao909]

【人工回复】【Manual reply】

“无实体卡时workload的生成”
可以参考 ，不开启 --aiob_enable(默认 fixed compute time)的相关命令

“指定网络结构与并行策略下的timeline估算”
网络结构是 topo

比如这个 ns3 后端
SimAI-Simulation (NS3) | gen_Topo_Template.py 生成 5 种模板拓扑

并行策略，应该还是workload时候指定， 比如切分到单卡上的 dp、 tp的计算量， 多卡通信量和通信域应该应该也在aiob里面，生成对应的workload 传到 后端 做通信仿真；
计算时间应该是aiob里面 真机测试 或者 人为赋值 获得

[tianhao909]

当前计算耗时仅支持通过aiob基于目标型号实体卡上的估算，未来是否支持无实体卡时workload的生成？以及是否支持指定网络结构与并行策略下的timeline估算？

[Reply from Agent A] [For reference only]

CN

Issue #285 回复:无实体卡时的 workload 生成与 timeline 估算

问题

当前计算耗时仅支持通过 aiob 基于目标型号实体卡上的估算,未来是否支持无实体卡时 workload 的生成?以及是否支持指定网络结构与并行策略下的 timeline 估算?

回复总览

感谢您的提问。以下两项能力 SimAI 主线已经支持,无需等待新功能:

1. 无实体卡(no physical GPU)生成 workload(工作负载描述文件):AICB(AI Communication Benchmark,阿里云通信基准生成器)在 CPU-only 环境即可生成 workload。AICB 提供三种 compute time(计算耗时)填充策略,只有一种会用到本机 GPU。
2. 指定 topology(网络拓扑)与 parallelism strategy(并行策略)的 timeline(时间线)估算:这是 SimAI-Analytical / SimAI-Simulation(NS3) 两个 frontend(前端)的核心能力,workload 首行声明并行策略,拓扑通过 busbw.yaml 或 gen_Topo_Template.py 模板表达,timeline 由 Layer::report() 输出 per-layer + 汇总 CSV。

下面分别详述。

---

一、关键术语对照

术语	中文	含义
SimAI-Analytical	SimAI 解析式仿真	用 busbw 抽象网络,跑得快,不依赖 GPU
SimAI-Simulation (NS3)	SimAI 高保真仿真(NS3 网络模拟器)	基于 NS3 的离散事件网络仿真,可拟合多种拓扑
SimAI-Physical (Beta)	SimAI 物理打流	在真实 RDMA 网卡上发包,不依赖 GPU,但需要 RDMA 网卡
AICB	AI Communication Benchmark	阿里云开源的 LLM 通信基准与 workload 生成器
AIOB	AI Operation Benchmark	AICB 的子模块,用于在实体 GPU 上 profile(画像)各计算 kernel(内核)耗时
workload	工作负载描述文件	文本文件,描述每层的 fp/ig/wg compute time、通信类型、消息大小、并行策略等
busbw	bus bandwidth(总线带宽)	一种简化的 collective(集合)通信带宽抽象,单位 GB/s
algbw	algorithm bandwidth(算法带宽)	集合通信算法层面的带宽
TP / DP / EP / PP / VPP / GA	Tensor / Data / Expert / Pipeline Parallelism, Virtual PP, Gradient Accumulation	张量并行 / 数据并行 / 专家并行 / 流水线并行 / 虚拟流水线 / 梯度累积
exposed comm	暴露通信(未被计算掩盖的通信耗时)	实际拖慢迭代的那部分通信
bubble	流水线气泡(空泡耗时)	PP 各 stage(段)间因依赖等待产生的空闲时间
collective communication	集合通信	AllReduce / AllGather / ReduceScatter / AlltoAll 等
frontend	前端	SimAI 适配不同网络模拟器的入口层
profile	画像 / 性能采样	在真实硬件上测量某个操作的实际耗时

---

二、整体链路

graph TB
    A[选择: 是否有目标型号 GPU] --> B[AICB Workload 生成阶段 - CPU-only 即可]

    B --> B1[路径 1: 默认 fixed compute time]
    B --> B2[路径 2: aiob_enable + comp_filepath 复用预生成 AIOB 文件]
    B --> B3[路径 3: 直接手工编辑 workload txt 的 compute_time 字段]
    B --> B4[路径 4 仅本机有目标 GPU: aiob_enable 现场 profile]

    B1 --> W[workload txt 文件]
    B2 --> W
    B3 --> W
    B4 --> W

    W --> S[SimAI 仿真层]

    S --> S1[SimAI-Analytical: busbw.yaml + 参数化拓扑 - 任意 server]
    S --> S2[SimAI-Simulation NS3: gen_Topo_Template.py 拓扑 - 任意 server]
    S --> S3[SimAI-Physical: RDMA 物理发包 - 需 RDMA 网卡]

    S1 --> R[Timeline 输出: EndToEnd CSV + SUM 行 + 可视化 HTML]
    S2 --> R
    S3 --> R

Loading

整条链路里 SimAI 仿真层不读取本机 GPU,workload txt 中的 compute_time 是离线值,见 Workload.cc#L1287-L1310:

Tick fp_compute_time;
inFile >> fp_compute_time;       // 直接从 workload txt 读取,与本机硬件无关
...
Tick ig_compute_time;
inFile >> ig_compute_time;
...
Tick wg_compute_time;
inFile >> wg_compute_time;

---

三、问题 1 详解:无实体卡如何生成 workload

AICB 官方 README 中明确写道(原文):

To generate workload traffic patterns for large model parallel framework training, you can use a CPU-only environment.

翻译:为大模型并行框架训练生成 workload 通信模式,CPU-only 环境即可。

具体地,AICB 提供 3 条无 GPU 路径 + 1 条需 GPU 路径:

路径 1:不开启 --aiob_enable(默认 fixed compute time)

适合快速搭建仿真链路或只关注通信侧的场景。AICB 会用一组固定的默认 compute time 填充 workload。命令示例(在 aicb 仓):

sh ./scripts/megatron_workload_with_aiob.sh \
  -m 7 --world_size 4096 \
  --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 8192 \
  --micro_batch 1 --seq_length 4096 \
  --swiglu --use_flash_attn
# 注意: 没有 --aiob_enable

路径 2:--aiob_enable --comp_filepath(复用他人 AIOB 输出)

如果你想用贴近真实 GPU的 compute time,但本机没有该型号 GPU,可以使用 AICB 仓库自带或社区贡献的 AIOB 输出文件:

sh ./scripts/megatron_workload_with_aiob.sh -m 7 \
  --world_size 4096 --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 8192 \
  --micro_batch 1 --seq_length 4096 --swiglu \
  --use_flash_attn --aiob_enable \
  --comp_filepath workload/aiob_inputs/Example.txt

workload/aiob_inputs/Example.txt 是 aicb 仓库提供的现成文件;如果你或同事在某型号 GPU 上跑过一次 AIOB,把得到的 results/aiob_outputs/*.txt 也可以直接作为 --comp_filepath 输入。

路径 3:手工编辑 workload txt

workload txt 的每一行格式如下,见 example/workload_analytical.txt:

layer_name  dependency  fp_compute_time  fp_comm_type  fp_comm_size \
            ig_compute_time  ig_comm_type  ig_comm_size \
            wg_compute_time  wg_comm_type  wg_comm_size  wg_update_time

示例第 9 行:

embedding_layer -1 622731 ALLREDUCE 50331648 1 NONE 0 15091072 NONE 0 100

其中 622731 / 1 / 15091072 三个数字就是 fp / ig / wg 的 compute time(单位 cycle,FREQ 由 SimAI 内部约定,默认 1 GHz,即 622731 cycle ≈ 622.7 微秒)。完全可以手填,即使是估算值,也足以驱动仿真。

路径 4(对比):本机有目标 GPU 时,用 AIOB 现场 profile

sh ./scripts/megatron_workload_with_aiob.sh -m 7 \
  --world_size 4096 --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 8192 \
  --micro_batch 1 --seq_length 4096 --swiglu \
  --use_flash_attn --aiob_enable
# 注意: --aiob_enable 但不带 --comp_filepath, 此时 AIOB 会用本机当前 GPU profile kernel 时间

仅这条路径需要本机有目标 GPU。

例外说明

aicb 2.1 版本在 README 的 "Latest News" 中提到:

A Hopper-architecture GPU is required to profile computation time with AIOB for DeepSeek and Qwen3 inference models.

翻译:为 DeepSeek 与 Qwen3 推理模型用 AIOB profile compute time,需要 Hopper 架构 GPU(原因是它们用了 DeepGEMM 等 SM90 专用库)。但这只影响想自己重新 profile 这两类推理 workload的场景,不影响训练 workload 与已有 AIOB 输出的复用。

---

四、问题 2 详解:指定网络结构与并行策略的 timeline 估算

4.1 并行策略如何指定

并行策略写在 workload txt 的首行,见 workload_analytical.txt#L1:

HYBRID_TRANSFORMER_FWD_IN_BCKWD model_parallel_NPU_group: 2 ep: 16 pp: 12 vpp: 8 ga: 24 all_gpus: 9216 checkpoints: 0 checkpoint_initiates: 0 pp_comm 50331648

字段	中文	含义
model_parallel_NPU_group	TP 组大小	张量并行 / Tensor Parallelism 组的 GPU 数
ep	EP 组大小	专家并行 / Expert Parallelism 组的 GPU 数
pp	PP 段数	流水线并行 / Pipeline Parallelism 切分段数
vpp	VPP 数	虚拟流水线 / Virtual Pipeline,用于减小 PP bubble(气泡)
ga	梯度累积步数	Gradient Accumulation 的 micro-batch 步数
all_gpus	总 GPU 数	整个训练 job 的 GPU 数;DP = all_gpus / (TP * PP)
pp_comm	PP 通信消息大小	用于估算 PP exposed comm

DP(Data Parallelism / 数据并行)是从这些参数推导出来的,SimAI 在 Layer.cc#L357-L371 中计算:

$$\mathrm{TP}=\texttt{model_parallel_NPU_group},\quad \mathrm{PP}=\texttt{pipeline_model_parallelism}$$

$$\mathrm{DP}=\frac{\texttt{all_gpus}}{\mathrm{TP}\cdot \mathrm{PP}},\quad \mathrm{EP}=\texttt{expert_parallel_npu_group}$$

每行 layer 的通信类型还能带后缀 _EP / _DP_EP 来指定该集合通信落在哪个并行组,例如 ALLGATHER_DP_EP 表示走 DP-EP 组的 AllGather。

4.2 网络结构如何指定

SimAI 提供三种 frontend,分别覆盖不同保真度需求:

Frontend	拓扑表达方式	是否需要本机 GPU	是否需要 RDMA 网卡
SimAI-Analytical	busbw.yaml + 命令行 -g/-g_p_s/-nv/-nic/-nic_t	否	否
SimAI-Simulation (NS3)	gen_Topo_Template.py 生成 5 种模板拓扑	否	否
SimAI-Physical	真实集群网卡	否	是

SimAI-Analytical 用法

参考 Tutorial.md "Analytical Simulation" 章节:

./bin/SimAI_analytical \
  -w example/workload_analytical.txt \
  -g 9216 -g_p_s 8 \
  -r test- \
  -busbw example/busbw.yaml

• -g:总 GPU 数(对应 workload 首行 all_gpus)
• -g_p_s:每台 server 的 GPU 数,即 scale-up 域大小(NVLink/NVSwitch 域)
• -busbw:busbw.yaml 文件,分别给 TP / DP / EP 通信组指定 AllReduce / AllGather / ReduceScatter / AlltoAll 的总线带宽
• 可选 overlap(重叠)参数:-dp_o / -tp_o / -ep_o / -pp_o,范围 [0.0, 1.0],控制各通信组与计算的重叠比

AstraParamParse.hh#L100-L168 列出全部参数,包括 -nv(NVLink 带宽)、-nic(网卡总线带宽)、-nic_t(网卡型号 cx7/bf3)、-g_type(GPU 型号 A100/H100/H800/H20 等)。

SimAI-Simulation (NS3) 用法

参考 Tutorial.md "TOPO Setting" 章节,先用 gen_Topo_Template.py 在 5 种模板中选一种生成拓扑文件:

• Spectrum-X
• DCN+ SingleToR
• DCN+ DualToR
• AlibabaHPN SinglePlane
• AlibabaHPN DualPlane

然后跑:

# 1. 编译 ns3 版本
./scripts/build.sh -c ns3

# 2. 生成拓扑文件(在 ns-3-alibabacloud 仓库 scripts 目录下)
python3 gen_Topo_Template.py -topo Spectrum-X -g 1024 ...

# 3. 运行仿真
./bin/SimAI_simulator -t 1 -w example/workload_analytical.txt -n <topo_file> -c <conf_file>

NS3 入口与拓扑解析见 entry.h、SetupNetwork 等函数。

4.3 timeline 输出长什么样

仿真完成后会在 -r 指定的目录下生成 EndToEnd CSV,核心由 Layer.cc#L337-L539 Layer::report() 写出。每行是一个 layer,列含义:

列	中文含义
fwd compute	正向计算耗时(forward pass compute)
wg compute	weight gradient(权重梯度) compute time
ig compute	input gradient(输入梯度) compute time
fwd exposed comm	正向暴露通信耗时(没被 compute 掩盖的部分)
wg exposed comm	wg 暴露通信耗时
ig exposed comm	ig 暴露通信耗时
fwd/wg/ig total comm + algbw + busbw	该层各阶段总通信量、算法带宽、总线带宽
workload finished at	此条 layer 在仿真时间线上的完成时刻

最后还有 SUM 行汇总,加一行 total exposed comm / total comp / total time,再加一行通信组分解(摘自 Layer.cc#L523-L535):

File name, Expose DP comm, Expose DP_EP comm, Expose TP comm, Expose_EP_comm,
Expose_PP_comm, bubble time, total comp, total exposed comm, Total time

每项还附百分比,直接给出"端到端一次迭代的 timeline 分解"。整体仿真时长另在 Sys::~Sys() 析构时打印,见 Sys.cc#L45-L72。

加 -v 1 还会生成可视化 HTML(对应 Tutorial.md "Visualization" 一节)。

---

五、最小验证脚本(全部不需要 GPU)

步骤 1:用 AICB 生成无 GPU workload

git clone https://github.com/aliyun/aicb.git
cd aicb
sh ./scripts/megatron_workload_with_aiob.sh \
  -m 7 --world_size 16 \
  --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 16 \
  --micro_batch 1 --seq_length 2048 \
  --swiglu --use_flash_attn
# 输出: results/mocked_workload/*.txt

步骤 2:用 SimAI-Analytical 仿真出 timeline

git clone https://github.com/aliyun/SimAI.git
cd SimAI
git submodule update --init --recursive
./scripts/build.sh -c analytical
./bin/SimAI_analytical \
  -w example/workload_analytical.txt \
  -g 9216 -g_p_s 8 \
  -r test- \
  -busbw example/busbw.yaml
# 输出: results/test-EndToEnd.csv 等

整套流程 不需要任何 GPU。

---

六、回答总结

您的问题	回答	主要证据
是否支持无实体卡时 workload 的生成?	当前已支持。 AICB CPU-only 环境即可,3 种无 GPU 路径(默认 fixed time / 复用 AIOB 输出 / 手工编辑)	aicb README, aicb tutorial.md, Workload.cc#L1287-L1310
是否支持指定网络结构与并行策略下的 timeline 估算?	当前已支持。 并行策略写 workload 首行,网络结构由 busbw.yaml 或 NS3 拓扑模板表达,timeline 由 Layer::report() 输出 per-layer + SUM CSV	workload_analytical.txt#L1, Layer.cc#L337-L539, AnalyticalAstra.cc#L68-L142

---

七、文档建议(可选)

Tutorial.md#L33 当前的描述强调"需要 GPU 环境跑 AIOB",容易误导认为没卡就跑不起来。建议在该段后追加一段 FAQ:

Note: 如果你只想生成 workload 或运行 SimAI-Analytical / SimAI-Simulation 仿真,而手头没有目标型号 GPU,可使用 AICB 的 --comp_filepath workload/aiob_inputs/Example.txt(复用预生成的 AIOB profile),或省略 --aiob_enable(用默认 fixed compute time);workload txt 中的 fp / ig / wg compute_time 字段也可手工编辑。SimAI 仿真层不依赖本机 GPU。

如果方便,欢迎社区共建多型号 GPU 的 AIOB profile 文件,一起放到 workload/aiob_inputs/ 下供他人复用。

---

如有进一步问题,例如需要某个特定模型 / 拓扑下的具体仿真示例、想让我们演示 gen_Topo_Template.py 的某个模板,或想了解 busbw 如何根据集群规模自动估算,欢迎在 issue 中继续讨论。

EN

Issue #285 Reply: Workload Generation Without a Physical GPU and Timeline Estimation

Question

Currently the compute time can only be estimated by running AIOB on a physical GPU of the target model. In the future, will workload generation be supported without a physical card? And will timeline estimation under a specified network topology and parallelism strategy be supported?

TL;DR

Thanks for the question. Both capabilities are already supported on the SimAI mainline, no new feature work is required:

1. Workload generation without a physical GPU is supported by AICB (AI Communication Benchmark, Alibaba Cloud's open-source benchmarking and workload generator) in a CPU-only environment. AICB offers three GPU-free strategies for filling in compute time, plus one optional path that needs a GPU.
2. Timeline estimation under a specified topology and parallelism strategy is the core function of two SimAI frontends, SimAI-Analytical and SimAI-Simulation (NS3). The parallelism strategy is declared on the first line of the workload file, the topology is described via busbw.yaml or gen_Topo_Template.py templates, and the timeline is emitted by Layer::report() as a per-layer + SUM CSV.

Details below.

---

1. Terminology

Term	Meaning
SimAI-Analytical	Fast simulator that abstracts the network with bus bandwidth (busbw); no GPU required
SimAI-Simulation (NS3)	High-fidelity simulator backed by the NS3 discrete-event network simulator
SimAI-Physical (Beta)	Generates real RDMA traffic; needs RDMA NICs but not GPUs
AICB	AI Communication Benchmark, the SimAI workload generator
AIOB	AI Operation Benchmark, AICB sub-module that profiles compute kernels on a real GPU
workload	Plain-text description file: per-layer fp / ig / wg compute time, communication type, message size, and parallelism strategy
busbw	Bus bandwidth (GB/s), a simplified collective-communication abstraction
algbw	Algorithm bandwidth, the bandwidth at the collective-algorithm level
TP / DP / EP / PP / VPP / GA	Tensor / Data / Expert / Pipeline parallelism, Virtual PP, Gradient Accumulation
exposed comm	Communication time not hidden by overlapped compute; the part that actually slows the iteration
bubble	Pipeline bubble, idle time between PP stages caused by dependencies
collective communication	AllReduce / AllGather / ReduceScatter / AlltoAll, etc.
frontend	SimAI's adapter layer to a specific network simulator
profile	To measure the real execution time of an operation on real hardware

---

2. End-to-End Pipeline

graph TB
    A[Decision: do you have a physical GPU of the target model] --> B[AICB workload generation - CPU-only is enough]

    B --> B1[Path 1: default fixed compute time]
    B --> B2[Path 2: aiob_enable + comp_filepath - reuse pre-generated AIOB output]
    B --> B3[Path 3: hand-edit compute_time fields in workload txt]
    B --> B4[Path 4 only if local GPU available: aiob_enable on-the-fly profiling]

    B1 --> W[workload txt file]
    B2 --> W
    B3 --> W
    B4 --> W

    W --> S[SimAI simulation layer]

    S --> S1[SimAI-Analytical: busbw.yaml + parametrized topology - any server]
    S --> S2[SimAI-Simulation NS3: gen_Topo_Template.py topology - any server]
    S --> S3[SimAI-Physical: real RDMA traffic - RDMA NIC required]

    S1 --> R[Timeline output: EndToEnd CSV + SUM row + visualization HTML]
    S2 --> R
    S3 --> R

Loading

The simulator itself does not query the local GPU. The compute_time fields in the workload file are offline values, see Workload.cc#L1287-L1310:

Tick fp_compute_time;
inFile >> fp_compute_time;       // read directly from the workload txt, hardware-agnostic
...
Tick ig_compute_time;
inFile >> ig_compute_time;
...
Tick wg_compute_time;
inFile >> wg_compute_time;

---

3. Question 1: How to Generate a Workload Without a GPU

The AICB README states verbatim:

To generate workload traffic patterns for large model parallel framework training, you can use a CPU-only environment.

AICB exposes three GPU-free paths plus one GPU-required path:

Path 1: Skip --aiob_enable (default fixed compute time)

Best for bringing up the simulation pipeline quickly or focusing only on the communication side. AICB fills the workload with a set of fixed default compute times. Example (in the aicb repo):

sh ./scripts/megatron_workload_with_aiob.sh \
  -m 7 --world_size 4096 \
  --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 8192 \
  --micro_batch 1 --seq_length 4096 \
  --swiglu --use_flash_attn
# Note: no --aiob_enable

Path 2: --aiob_enable --comp_filepath (reuse a pre-generated AIOB file)

If you want compute times that resemble a real GPU but you do not own one, reuse an AIOB output that someone else produced. AICB ships sample files under workload/aiob_inputs/:

sh ./scripts/megatron_workload_with_aiob.sh -m 7 \
  --world_size 4096 --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 8192 \
  --micro_batch 1 --seq_length 4096 --swiglu \
  --use_flash_attn --aiob_enable \
  --comp_filepath workload/aiob_inputs/Example.txt

Any results/aiob_outputs/*.txt produced by a prior run on the target GPU can also be used as --comp_filepath.

Path 3: Hand-edit the workload txt

Each line of the workload file follows this layout, see example/workload_analytical.txt:

layer_name  dependency  fp_compute_time  fp_comm_type  fp_comm_size \
            ig_compute_time  ig_comm_type  ig_comm_size \
            wg_compute_time  wg_comm_type  wg_comm_size  wg_update_time

Line 9 of the example file:

embedding_layer -1 622731 ALLREDUCE 50331648 1 NONE 0 15091072 NONE 0 100

The numbers 622731 / 1 / 15091072 are the fp / ig / wg compute times in cycles (FREQ is fixed inside SimAI, defaulting to 1 GHz, so 622731 cycles ~= 622.7 microseconds). You can simply fill these by hand; even rough estimates are enough to drive the simulator.

Path 4 (for comparison): on-the-fly profiling on a local GPU

sh ./scripts/megatron_workload_with_aiob.sh -m 7 \
  --world_size 4096 --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 8192 \
  --micro_batch 1 --seq_length 4096 --swiglu \
  --use_flash_attn --aiob_enable
# Note: --aiob_enable but no --comp_filepath, so AIOB profiles kernels on the local GPU.

This is the only path that requires a physical GPU.

Caveat

The aicb 2.1 release notes mention:

A Hopper-architecture GPU is required to profile computation time with AIOB for DeepSeek and Qwen3 inference models.

This applies only when you want to freshly profile DeepSeek or Qwen3 inference workloads (because they rely on DeepGEMM / SM90-only libraries). Training workloads and reusing existing AIOB outputs are not affected.

---

4. Question 2: Timeline Estimation Under a Specified Topology and Parallelism Strategy

4.1 Specifying the parallelism strategy

The parallelism strategy lives on the first line of the workload txt, see workload_analytical.txt#L1:

HYBRID_TRANSFORMER_FWD_IN_BCKWD model_parallel_NPU_group: 2 ep: 16 pp: 12 vpp: 8 ga: 24 all_gpus: 9216 checkpoints: 0 checkpoint_initiates: 0 pp_comm 50331648

Field	Meaning
model_parallel_NPU_group	TP group size (Tensor Parallelism)
ep	EP group size (Expert Parallelism)
pp	Number of PP stages (Pipeline Parallelism)
vpp	Virtual PP, used to reduce PP bubble
ga	Gradient accumulation steps (number of micro-batches)
all_gpus	Total GPUs in the job; DP = all_gpus / (TP * PP)
pp_comm	Per-stage PP message size, used to estimate exposed PP comm

DP is derived from these fields. SimAI computes it in Layer.cc#L357-L371:

$$\mathrm{TP}=\texttt{model_parallel_NPU_group},\quad \mathrm{PP}=\texttt{pipeline_model_parallelism}$$

$$\mathrm{DP}=\frac{\texttt{all_gpus}}{\mathrm{TP}\cdot \mathrm{PP}},\quad \mathrm{EP}=\texttt{expert_parallel_npu_group}$$

The communication-type column on each layer line can carry a suffix such as _EP or _DP_EP to bind the collective to a specific group, e.g. ALLGATHER_DP_EP runs an AllGather over the DP-EP group.

4.2 Specifying the network topology

SimAI provides three frontends for different fidelity needs:

Frontend	How topology is described	Local GPU required	RDMA NIC required
SimAI-Analytical	busbw.yaml + flags -g/-g_p_s/-nv/-nic/-nic_t	No	No
SimAI-Simulation (NS3)	gen_Topo_Template.py generates one of five template topologies	No	No
SimAI-Physical	Real cluster NICs	No	Yes

SimAI-Analytical command

See the "Analytical Simulation" section of Tutorial.md:

./bin/SimAI_analytical \
  -w example/workload_analytical.txt \
  -g 9216 -g_p_s 8 \
  -r test- \
  -busbw example/busbw.yaml

• -g: total GPU count (matches all_gpus on the workload first line)
• -g_p_s: GPUs per server, i.e. the scale-up domain size (the NVLink/NVSwitch domain)
• -busbw: busbw.yaml, giving the bus bandwidth for AllReduce / AllGather / ReduceScatter / AlltoAll on each of the TP / DP / EP groups
• Optional overlap flags -dp_o / -tp_o / -ep_o / -pp_o in [0.0, 1.0] control how much each communication group overlaps with compute

AstraParamParse.hh#L100-L168 lists the full flag set, including -nv (NVLink bandwidth), -nic (NIC bus bandwidth), -nic_t (NIC type cx7/bf3), and -g_type (A100/H100/H800/H20 etc.).

SimAI-Simulation (NS3) command

See the "TOPO Setting" section of Tutorial.md. Use gen_Topo_Template.py to pick one of five templates:

• Spectrum-X
• DCN+ SingleToR
• DCN+ DualToR
• AlibabaHPN SinglePlane
• AlibabaHPN DualPlane

Then:

# 1. Build the ns3 variant
./scripts/build.sh -c ns3

# 2. Generate a topology file (in the ns-3-alibabacloud repo, scripts dir)
python3 gen_Topo_Template.py -topo Spectrum-X -g 1024 ...

# 3. Run the simulation
./bin/SimAI_simulator -t 1 -w example/workload_analytical.txt -n <topo_file> -c <conf_file>

The NS3 entry point and topology parsing live in entry.h (SetupNetwork and friends).

4.3 What the timeline output looks like

After simulation, the EndToEnd CSV is written to the directory pointed to by -r. The core is in Layer.cc#L337-L539 (Layer::report()). Each row is a layer; the columns are:

Column	Meaning
fwd compute	Forward-pass compute time
wg compute	Weight-gradient compute time
ig compute	Input-gradient compute time
fwd exposed comm	Forward-pass exposed communication (the part not hidden by compute)
wg exposed comm	wg exposed communication
ig exposed comm	ig exposed communication
fwd/wg/ig total comm + algbw + busbw	Total per-stage comm volume, algorithm bandwidth, bus bandwidth
workload finished at	The point on the simulated timeline when this layer finished

A SUM row aggregates all layers, followed by a total exposed comm / total comp / total time line and a final per-group breakdown line (see Layer.cc#L523-L535):

File name, Expose DP comm, Expose DP_EP comm, Expose TP comm, Expose_EP_comm,
Expose_PP_comm, bubble time, total comp, total exposed comm, Total time

Each entry is also reported as a percentage, giving a ready-to-use timeline breakdown of one end-to-end iteration. The total simulation duration is also printed in the Sys::~Sys() destructor, see Sys.cc#L45-L72.

Adding -v 1 produces a visualization HTML (see the "Visualization" section of Tutorial.md).

---

5. Minimal GPU-Free Walkthrough

Step 1: Generate a workload with AICB (no GPU)

git clone https://github.com/aliyun/aicb.git
cd aicb
sh ./scripts/megatron_workload_with_aiob.sh \
  -m 7 --world_size 16 \
  --tensor_model_parallel_size 2 --pipeline_model_parallel 1 \
  --frame Megatron --global_batch 16 \
  --micro_batch 1 --seq_length 2048 \
  --swiglu --use_flash_attn
# Output: results/mocked_workload/*.txt

Step 2: Run SimAI-Analytical to get a timeline

git clone https://github.com/aliyun/SimAI.git
cd SimAI
git submodule update --init --recursive
./scripts/build.sh -c analytical
./bin/SimAI_analytical \
  -w example/workload_analytical.txt \
  -g 9216 -g_p_s 8 \
  -r test- \
  -busbw example/busbw.yaml
# Output: results/test-EndToEnd.csv etc.

The whole flow does not need any GPU.

---

6. Summary

Your question	Answer	Primary evidence
Will workload generation be supported without a physical GPU?	Already supported. AICB runs in a CPU-only environment with three GPU-free paths (default fixed time, reuse AIOB output, hand-edit).	aicb README, aicb tutorial.md, Workload.cc#L1287-L1310
Will timeline estimation under a specified topology and parallelism strategy be supported?	Already supported. The parallelism strategy is on the workload first line, the topology is described via busbw.yaml or NS3 topology templates, and the timeline is a per-layer + SUM CSV emitted by Layer::report().	workload_analytical.txt#L1, Layer.cc#L337-L539, AnalyticalAstra.cc#L68-L142

---

7. Documentation Suggestion (Optional)

The current text at Tutorial.md#L33 emphasizes "AIOB needs a GPU environment", which can mislead readers into thinking a GPU is mandatory. We suggest appending a small FAQ note:

Note: If you only need to generate workloads or run SimAI-Analytical / SimAI-Simulation simulations and you do not have the target GPU, use AICB's --comp_filepath workload/aiob_inputs/Example.txt (reuses a pre-generated AIOB profile) or simply omit --aiob_enable (uses default fixed compute times). The fp / ig / wg compute_time fields in the workload txt can also be filled by hand. The SimAI simulation layer does not depend on the local GPU.

Community contributions of AIOB profile files for additional GPU models are also welcome under workload/aiob_inputs/ so others can reuse them.

---

If you have follow-up needs, e.g. a concrete example for a specific model + topology, a walkthrough of a particular gen_Topo_Template.py template, or guidance on how to derive busbw automatically for a given cluster, please continue the discussion on this issue.

[tianhao909]

[Reply from Agent B] [For reference only]

CN

感谢反馈。这个问题可以分成两个层面回答：

1. 无实体卡时是否支持 workload 生成？

目前已经可以部分支持。

• 如果不启用 --aiob_enable，AICB 的 SimAI workload 生成器会使用默认固定 compute time 生成 workload。这种方式不依赖目标 GPU，适合做功能验证、通信模式分析、并行策略对比和 SimAI 流程打通。
• 如果已经有目标硬件上的 AIOB profiling 输出，也可以通过 --aiob_enable --comp_filepath <file> 复用已有 compute profile。在这种模式下，本机不需要再有目标型号实体卡。
• 如果需要得到接近目标 GPU 的真实计算耗时，仍然建议在目标硬件上使用 AIOB 采集，或者使用可信的离线 profile / 经验模型。没有任何目标硬件数据时，compute time 只能使用默认值或近似模型，精度会受限。

因此，结论是：无实体卡可以生成 workload，但计算耗时精度取决于是否有可复用的 profiling 数据或离线模型；默认模式不等价于目标卡真实计算耗时。

2. 是否支持指定网络结构与并行策略下的 timeline 估算？

目前也已经有基础能力，但入口相对分散。

• 并行策略主要由 AICB workload 生成参数决定，例如 TP、PP、EP、world size、global batch、micro batch、sequence length 等；这些信息会写入 SimAI workload 文件头。
• 网络结构由 SimAI-Simulation 的 topology 文件和 config 文件指定。
• SimAI-Analytical 可以用于快速估算通信时间；SimAI-Simulation/ns-3 可以用于更完整的网络仿真。
• Vidur 集成路径中也支持通过 backend 选择 simai_analytical、simai_simulation 或 aicb，并指定 simai_simulation_topo、simai_simulation_config 来做多请求推理场景下的 timeline / trace 估算。

限制是：如果希望从“任意模型结构 + 任意并行策略”自动生成高精度 timeline，则还需要该模型结构在 AICB 中有对应的 layer/communication 建模，并且 compute time 需要来自 AIOB profile、已有 comp_filepath、默认值或离线经验模型。

后续建议

我们建议先补充文档，明确以下几种模式：

1. 无 AIOB / 无实体卡模式：不传 --aiob_enable，使用默认 compute time 生成 workload。
2. 离线 profile 模式：传 --aiob_enable --comp_filepath <existing_aiob_output>，复用已有 profiling 数据。
3. 真实硬件 profiling 模式：在目标 GPU 上使用 AIOB 采集 compute time。
4. timeline 估算流程：AICB 生成 workload → SimAI-Analytical 或 SimAI-Simulation 指定 topology/config 运行 → 查看 end-to-end CSV / detailed trace / chrome trace。

如果需要进一步增强代码，最小改动可以是增加一个 --default_compute_time 参数，让无实体卡模式下的默认 compute time 可配置，同时保持现有默认行为不变。

EN

Thanks for the question. I would split it into two parts:

1. Can workload generation be supported without a physical target GPU?

It is already partially supported today.

• If --aiob_enable is not enabled, the AICB SimAI workload generator uses a fixed default compute time to generate the workload. This does not require a target GPU and is useful for functional validation, communication pattern analysis, parallelism comparison, and end-to-end SimAI workflow testing.
• If an AIOB profiling output from the target hardware already exists, it can be reused with --aiob_enable --comp_filepath <file>. In this mode, the local machine does not need to have the target GPU.
• If accurate compute time for a specific target GPU is required, we still recommend collecting AIOB profiling data on the target hardware, or using a trusted offline profile / empirical model. Without any target-hardware data, the compute time can only come from a default value or an approximate model, so accuracy will be limited.

So the short answer is: yes, workload generation without a physical GPU is possible, but compute-time accuracy depends on whether reusable profiling data or an offline model is available. The default mode should not be interpreted as real target-GPU compute time.

2. Is timeline estimation supported for a specified network topology and parallelism strategy?

The basic capability already exists, although the current entry points are relatively scattered.

• The parallelism strategy is mainly determined by AICB workload generation parameters, such as TP, PP, EP, world size, global batch size, micro batch size, and sequence length. These parameters are written into the SimAI workload header.
• The network topology is specified through the SimAI-Simulation topology file and config file.
• SimAI-Analytical can be used for fast communication-time estimation, while SimAI-Simulation/ns-3 can be used for more complete network simulation.
• The Vidur integration path can also select simai_analytical, simai_simulation, or aicb as the backend, and can specify simai_simulation_topo and simai_simulation_config for multi-request inference timeline / trace estimation.

The limitation is that fully automatic high-accuracy timeline generation for an arbitrary model structure and arbitrary parallel strategy still requires the model/layer/communication pattern to be supported by AICB, and compute time must come from AIOB profiling, an existing comp_filepath, a default value, or an offline empirical model.

Suggested next steps

We suggest first improving the documentation to clearly describe the following modes:

1. No-AIOB / no-physical-GPU mode: do not pass --aiob_enable; use default compute time to generate the workload.
2. Offline profile mode: pass --aiob_enable --comp_filepath <existing_aiob_output> to reuse existing profiling data.
3. Real hardware profiling mode: run AIOB on the target GPU to collect compute time.
4. Timeline estimation flow: generate workload with AICB → run SimAI-Analytical or SimAI-Simulation with the selected topology/config → inspect end-to-end CSV, detailed trace, or chrome trace.

If we want a small code enhancement, the minimal change would be to add a --default_compute_time option so that the no-physical-GPU mode can use a configurable default compute time while keeping the current default behavior unchanged.