scx_layered: skip absent CPUs when initializing per-cpu ctxs - #3718
Open
hodgesds wants to merge 1 commit into
Open
scx_layered: skip absent CPUs when initializing per-cpu ctxs#3718hodgesds wants to merge 1 commit into
hodgesds wants to merge 1 commit into
Conversation
init_cpus() iterated 0..NR_CPUS_POSSIBLE and unwrapped topo.all_cpus.get(&cpu), which panics for possible-but-not-present CPUs. On bare metal possible == online so this never triggers, but a VM booted on a kernel with CONFIG_NR_CPUS > vCPU count (e.g. an fbk kernel with NR_CPUS=512 booted with 384 vCPUs) has a sparse possible range and hits the panic at the first missing slot. Guard the topo lookup and skip absent slots. The percpu map still gets one entry per possible CPU (the ctx stays zero-initialized for absent slots, which is fine because BPF never runs on them), so downstream code that indexes cpu_ctxs[cpu_id] keeps working. Validated with virtme-run using --cpus 384 --qemu-opts -smp cpus=384,maxcpus=512 (KVM capped online at 256, so the actual guest saw possible=0-511 online=0-255): Before (panic): VM_UP POSSIBLE=0-511 ONLINE=0-255 INFO scx_layered: CPUs: online/possible=256/512 nr_cores=256 INFO scx_layered: Running scx_layered (build ID: 1.1.2 x86_64-unknown-linux-gnu/debug) thread 'main' (3014) panicked at scheds/rust/scx_layered/src/main.rs:2428:52: called `Option::unwrap()` on a `None` value EXITCODE=101 After (clean startup and shutdown): VM_UP POSSIBLE=0-511 ONLINE=0-255 INFO scx_layered: CPUs: online/possible=256/512 nr_cores=256 INFO scx_layered: Running scx_layered (build ID: 1.1.2 x86_64-unknown-linux-gnu/debug) INFO scx_layered: Layered Scheduler Attached. Run `scx_layered --monitor` for metrics. EXIT: unregistered from user space INFO scx_layered: Unregister scx_layered scheduler EXITCODE=0
Contributor
|
I think that's a bug in the VMM if it happens right? Mentioning this because I addressed this in ktstr a week or two ago while productionizing things to enable use here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
init_cpus() iterated 0..NR_CPUS_POSSIBLE and unwrapped topo.all_cpus.get(&cpu), which panics for possible-but-not-present CPUs. On bare metal possible == online so this never triggers, but a VM booted on a kernel with CONFIG_NR_CPUS > vCPU count (e.g. an fbk kernel with NR_CPUS=512 booted with 384 vCPUs) has a sparse possible range and hits the panic at the first missing slot.
Guard the topo lookup and skip absent slots. The percpu map still gets one entry per possible CPU (the ctx stays zero-initialized for absent slots, which is fine because BPF never runs on them), so downstream code that indexes cpu_ctxs[cpu_id] keeps working.
Validated with virtme-run using --cpus 384 --qemu-opts -smp cpus=384,maxcpus=512 (KVM capped online at 256, so the actual guest saw possible=0-511 online=0-255):
Before (panic):
VM_UP
POSSIBLE=0-511
ONLINE=0-255
INFO scx_layered: CPUs: online/possible=256/512 nr_cores=256
INFO scx_layered: Running scx_layered (build ID: 1.1.2 x86_64-unknown-linux-gnu/debug)
thread 'main' (3014) panicked at scheds/rust/scx_layered/src/main.rs:2428:52:
called
Option::unwrap()on aNonevalueEXITCODE=101
After (clean startup and shutdown):
VM_UP
POSSIBLE=0-511
ONLINE=0-255
INFO scx_layered: CPUs: online/possible=256/512 nr_cores=256
INFO scx_layered: Running scx_layered (build ID: 1.1.2 x86_64-unknown-linux-gnu/debug)
INFO scx_layered: Layered Scheduler Attached. Run
scx_layered --monitorfor metrics.EXIT: unregistered from user space
INFO scx_layered: Unregister scx_layered scheduler
EXITCODE=0