Centralize guest physical layout logic by jstarks · Pull Request #3501 · microsoft/openvmm

jstarks · 2026-05-16T17:39:05Z

Layout decisions ("where does this MMIO range live?") are currently spread across the command line, ttrpc, petri, and the worker, each carving up DEFAULT_MMIO_GAPS_* in its own way. This PR replaces all of that with a single allocator in vm_topology::layout and a worker resolver in openvmm_core::worker::memory_layout. Front ends now specify sizes; the resolver picks addresses.

What changes for callers:

Sizes are still configurable (chipset MMIO, per–root-complex PCIe windows, VTL2).
PCIe BARs can still take a fixed range to support physical topology passthrough.
Addresses are otherwise no longer configurable.

Improvements as part of this:

virtio-mmio has its own dedicated reservation, sized to the slot count, instead of stacking at the top of an MMIO gap.
VTL2 (chipset MMIO and IGVM private memory) is placed deterministically above all VTL0-visible layout, so enabling VTL2 cannot shift any VTL0 address.

See memory-layout.md for the allocator phases, worker policy, and worked examples.

Add a pure-math layout allocator in vm_topology::layout that assigns address ranges to MMIO consumers within a flat physical address space. The allocator has no knowledge of specific architectures, firmware types, or pinned-address conventions -- those are the responsibility of the caller. Consumers call request() to declare allocation needs, passing a &mut MemoryRange that will be filled in when allocate() runs. Pinned requests are placed first at their fixed addresses with overlap validation. Dynamic requests (Below4GiB / Above4GiB) are sorted by alignment desc, size desc, input order asc, then greedy top-down placed into the remaining free space. allocate() returns a sorted Vec<MemoryRange> of all allocations. Free-space tracking uses subtract_ranges for linear-time updates. Pinned overlap detection sorts by address and checks adjacent pairs.

Copilot

Pull request overview

This PR centralizes guest physical address layout decisions into vm_topology::layout and the OpenVMM worker resolver, replacing per-frontend MMIO gap carving with size-based layout requests.

Changes:

Adds a deterministic layout allocator and worker memory-layout resolver.
Updates OpenVMM/Petri/ttrpc configuration to request MMIO sizes instead of fixed default gaps.
Updates loader paths, PCIe/virtio placement, tests, and Guide documentation for the new layout model.

Reviewed changes

Copilot reviewed 23 out of 24 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`vmm_tests/vmm_tests/tests/tests/x86_64/openhcl_linux_direct.rs`	Updates VTL2 MMIO self-allocation test expectations.
`vmm_tests/vmm_tests/tests/tests/multiarch/memstat.rs`	Removes explicit Petri MMIO gap selection.
`vmm_core/vm_manifest_builder/src/lib.rs`	Adds default chipset MMIO sizing outputs.
`vm/vmcore/vm_topology/src/memory.rs`	Adds constructor for resolved layouts and adjusts layout end handling.
`vm/vmcore/vm_topology/src/lib.rs`	Exposes the new layout module.
`vm/vmcore/vm_topology/src/layout.rs`	Adds the generic layout allocator and tests.
`petri/src/vm/openvmm/modify.rs`	Converts PCIe test topology setup to dynamic MMIO window sizes.
`petri/src/vm/openvmm/construct.rs`	Removes Petri-side default MMIO gap construction.
`petri/src/vm/mod.rs`	Removes Petri `MmioConfig`.
`petri/Cargo.toml`	Removes unused `memory_range` dependency.
`openvmm/openvmm_entry/src/ttrpc/mod.rs`	Removes fixed MMIO defaults from ttrpc VM config.
`openvmm/openvmm_entry/src/lib.rs`	Removes CLI-side MMIO carving and emits dynamic PCIe sizing.
`openvmm/openvmm_entry/Cargo.toml`	Removes unused `memory_range` dependency.
`openvmm/openvmm_defs/src/config.rs`	Changes VM config schema to MMIO sizes and dynamic/fixed PCIe BAR windows.
`openvmm/openvmm_defs/Cargo.toml`	Enables mesh support for `memory_range`.
`openvmm/openvmm_core/src/worker/vm_loaders/linux.rs`	Uses resolved layout MMIO for Linux direct DT VMBus ranges.
`openvmm/openvmm_core/src/worker/vm_loaders/igvm.rs`	Moves VTL2 memory placement into layout request/resolution flow.
`openvmm/openvmm_core/src/worker/mod.rs`	Adds worker memory-layout module.
`openvmm/openvmm_core/src/worker/memory_layout.rs`	Adds worker policy resolver for RAM, MMIO, PCIe, virtio, and VTL2 ranges.
`openvmm/openvmm_core/src/worker/dispatch.rs`	Integrates resolved layout outputs into VM initialization, PCIe, virtio, ACPI, and IGVM paths.
`openhcl/underhill_core/src/worker.rs`	Ignores new manifest result fields where not used.
`Guide/src/SUMMARY.md`	Adds the new memory-layout Guide page.
`Guide/src/reference/architecture/openvmm/memory-layout.md`	Documents allocator phases and worker layout policy.
`Cargo.lock`	Updates dependency graph after dependency removals.

Comments suppressed due to low confidence (2)

openvmm/openvmm_core/src/worker/memory_layout.rs:275

Flattening RAM extents by vnode order can produce an unsorted ram vector even though MemoryLayout::new_from_resolved_ranges requires RAM ranges to be sorted by address. For example, a large 1 GiB-aligned vnode can skip a low fragment around a fixed MMIO hole, and a later smaller vnode can then allocate that lower fragment; validation will reject the otherwise valid layout. Preserve the vnode metadata but sort the flattened RAM ranges by address before constructing MemoryLayout.

    let ram = ram_ranges_by_node
        .into_iter()
        .enumerate()
        .flat_map(|(vnode, ranges)| {
            ranges.into_iter().map(move |range| MemoryRangeWithNode {
                range,
                vnode: vnode as u32,
            })
        })
        .collect::<Vec<_>>();

openvmm/openvmm_core/src/worker/memory_layout.rs:327

This host-width check only uses MemoryLayout::end_of_layout(), which explicitly excludes the optional VTL2 private range and does not include the returned vtl2_chipset_mmio range. As a result, a layout can pass validation even when VTL2 private memory or VTL2 chipset MMIO is placed above the host physical-address limit, leading to a later mapping/boot failure. Include all resolved ranges (including vtl2_range and vtl2_chipset_mmio) when validating the address-space limit.

    let address_space_limit = 1u64 << input.physical_address_size;
    if memory_layout.end_of_layout() > address_space_limit {

Copilot

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 5 comments.

github-actions · 2026-05-16T22:11:42Z

11 Petri tests failed (0 unstable)

Copilot

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

openvmm/openvmm_core/src/worker/memory_layout.rs:328

For a full bus window (start_bus = 0, end_bus = 255, which is the CLI default), this addition is performed as u8 before conversion to u64. 255 + 1 overflows (panics in checked builds and wraps to 0 in release), so the resolved per-root-complex ECAM range has the wrong length and GenericPcieRootComplex::new will later reject/panic because it expects 256 MiB. Widen the operands before adding one, as the segment ECAM calculation above does.

        let size =
            u64::from(root_complex.end_bus - root_complex.start_bus + 1) * PCIE_ECAM_BYTES_PER_BUS;
        ranges.ecam_range =
            MemoryRange::new(se.range.start() + offset..se.range.start() + offset + size);

+    if let Some(vtl2_layout) = input.vtl2_layout {
+        builder.request(
+            "vtl2",
+            &mut vtl2_range,
+            vtl2_layout.size,
+            vtl2_layout.alignment,
+            Placement::PostMmio,
+        );


+
+After `allocate()` succeeds, the worker collects the resolved ranges into
+the `MemoryLayout`'s MMIO, PCI ECAM, and PCI MMIO gap vectors, then checks
+`MemoryLayout::end_of_layout()` against the host's physical-address width.


+        if let Some(entry) = segment_ecams.iter_mut().find(|e| e.segment == rc.segment) {
+            entry.min_bus = entry.min_bus.min(rc.start_bus);
+            entry.max_bus = entry.max_bus.max(rc.end_bus);
+        } else {


github-actions · 2026-05-18T00:20:33Z

65 Petri tests failed (0 unstable)

github-actions · 2026-05-18T02:20:39Z

6 Petri tests failed (0 unstable)

Copilot

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 6 comments.

+    // Default chipset MMIO sizes for `HclHost` from
+    // `vm_manifest_builder::VmChipsetBuilder::build`. Keep in sync with that
+    // file.
+    const DEFAULT_LOW_MMIO_SIZE: u64 = 96 * 1024 * 1024;


+    /// The RAM, MMIO, PCI ECAM, and PCI MMIO ranges must each be in sorted
+    /// order, non-empty, and non-overlapping. The combined layout is also


+        let bus_count = u64::from(se.max_bus - se.min_bus) + 1;
+        builder.request(
+            format!("pcie-seg{}-ecam", se.segment),
+            &mut se.range,
+            bus_count * PCIE_ECAM_BYTES_PER_BUS,


+        max_bus: u8,
+        range: MemoryRange,
+    }
+    let mut segment_ecams: Vec<SegmentEcam> = Vec::new();
+    for rc in input.pcie_root_complexes {
+        if let Some(entry) = segment_ecams.iter_mut().find(|e| e.segment == rc.segment) {
+            entry.min_bus = entry.min_bus.min(rc.start_bus);
+            entry.max_bus = entry.max_bus.max(rc.end_bus);
+        } else {
+            segment_ecams.push(SegmentEcam {
+                segment: rc.segment,
+                min_bus: rc.start_bus,
+                max_bus: rc.end_bus,


+the `MemoryLayout`'s MMIO, PCI ECAM, and PCI MMIO gap vectors, then checks
+`MemoryLayout::end_of_layout()` against the host's physical-address width.


+                arch_reserved.end(),
+                arch_reserved.len(),
+            );
+        }


github-actions · 2026-05-18T05:33:29Z

3 Petri tests failed (0 unstable)

jstarks added 15 commits May 15, 2026 22:18

vm_topology: rework layout allocator around RAM

888a96b

Route worker RAM layout through allocator

35dcf39

Clarify VTL2 RAM layout validation

347fa63

vtl2 stuff

18793c4

cleanup

289b13e

wip

b07d719

cleanup

40cb3e4

docs

0f80677

pci

9e5c488

virtio-mmio

3f62efc

wip

23477ca

feedback

4804144

better guide

b575368

improve

af7d1a9

Copilot AI review requested due to automatic review settings May 16, 2026 17:39

jstarks requested a review from a team as a code owner May 16, 2026 17:39

github-actions Bot added the Guide label May 16, 2026