mitosis: skip foreign cgroup hierarchies in tp_cgroup_mkdir - #3606
Open
likewhatevs wants to merge 2 commits into
Open
mitosis: skip foreign cgroup hierarchies in tp_cgroup_mkdir#3606likewhatevs wants to merge 2 commits into
likewhatevs wants to merge 2 commits into
Conversation
For-realsies crash within ms of scheduler attach on a host with
cgroup activity outside the configured --cell-parent-cgroup
subtree:
scx_bpf_error (./mitosis.bpf.c:148: cgrp_ctx lookup failed for
cgid 1)
Backtrace:
bpf_prog_init_cgrp_ctx
bpf_prog_init_cgrp_ctx_with_ancestors
bpf_prog_tp_cgroup_mkdir
bpf_trace_run2
cgroup_mkdir
mitosis_init only initializes cgrp_ctx storage for the cgroupv2
default root via bpf_cgroup_from_id(root_cgid=1), which resolves
through cgrp_dfl_root.kf_root. When any cgroup is created in a
cgroupv1 named hierarchy (or any other non-default cgroup
hierarchy), the v1 root cgroup also reports kn->id = 1 (first
allocated in its own kernfs_root's idr) but its cgrp_ctx was
never initialized — it is a different cgroup struct than the v2
root.
tp_cgroup_mkdir fires for the v1 child. init_cgrp_ctx_with_ancestors
walks the chain. The level-1 ancestor's init_cgrp_ctx calls
lookup_cgrp_ctx(parent_cg) at mitosis.bpf.c:1510 where parent_cg
is the v1 root. Lookup returns NULL and the non-fallible wrapper
bails the scheduler at mitosis.bpf.c:148.
Reproducer mounts a per-worker named v1 hierarchy and mkdir+rmdir
level-2 cgroups in it. The bail fires reliably within ~2 s of
scheduler attach with the for-realsies backtrace.
Signed-off-by: Pat Somaru <patso@likewhatevs.io>
tp_btf/cgroup_mkdir fires for every cgroup created system-wide, not just cgroups in the configured --cell-parent-cgroup subtree and not just cgroupv2. When any cgroup is created in a foreign cgroup hierarchy (cgroupv1 named hierarchy, cgroupv1 subsystem mount, or any other non-default cgroup hierarchy), the handler's init_cgrp_ctx_with_ancestors walks the foreign-hierarchy ancestor chain. The walk's first level-1-ancestor init_cgrp_ctx call hits lookup_cgrp_ctx(parent_cg) where parent_cg is the foreign hierarchy's root cgroup. That root has no cgrp_ctx storage — mitosis_init only initialized the cgroupv2 default root via bpf_cgroup_from_id(root_cgid=1), which searches cgrp_dfl_root.kf_root — so the non-fallible lookup_cgrp_ctx wrapper bails the scheduler: scx_bpf_error (./mitosis.bpf.c:148: cgrp_ctx lookup failed for cgid 1) mitosis already ignores foreign hierarchies everywhere else: the mitosis_init descendants walk iterates only descendants of root_cgrp; mitosis_init_task reads task_cgroup() which returns the v2 unified cgroup; the SCX cgroup struct_ops callbacks only fire for v2. tp_cgroup_mkdir is the lone path that sees foreign cgroups, so the skip is local to this handler: when cgrp's top-of-hierarchy struct pointer differs from root_cgrp's, skip init_cgrp_ctx_with_ancestors entirely. The check uses bpf_cgroup_ancestor(cgrp, 0) to get the actual struct cgroup of cgrp's hierarchy root and compares pointer equality against an explicit bpf_cgroup_acquire(root_cgrp) inside an rcu-guarded block. Gated behind a new --fix-ignore-cg-v1 CLI flag (off by default, plumbed through rodata.fix_ignore_cg_v1) so the for-realsies bail is still observable until the operator opts in. With the flag set ktstr_mitosis_cgrp_ctx_lookup_v1_root passes; without it the test still fails with the bail backtrace, demonstrating the bug. Signed-off-by: Pat Somaru <patso@likewhatevs.io>
likewhatevs
force-pushed
the
mitosis-cgrp-ctx-lookup-extra-mkdir-repro
branch
from
May 28, 2026 05:32
b89de5d to
f6550f2
Compare
likewhatevs
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tp_btf/cgroup_mkdirfires for every cgroup created system-wide, not just cgroups in--cell-parent-cgroupand not just cgroupv2. When a cgroup is created in any foreign cgroup hierarchy (cgroupv1 named hierarchy, cgroupv1 subsystem mount, or any non-default cgroup hierarchy),init_cgrp_ctx_with_ancestorswalks the foreign-hierarchy ancestor chain. The first level-1-ancestorinit_cgrp_ctxcall hitslookup_cgrp_ctx(parent_cg)whereparent_cgis the foreign hierarchy's root cgroup. That root has nocgrp_ctxstorage —mitosis_initonly initialized the cgroupv2 default root viabpf_cgroup_from_id(root_cgid=1), which searchescgrp_dfl_root.kf_root— so the non-falliblelookup_cgrp_ctxwrapper bails:Two commits:
tp_cgroup_mkdirthat comparescgrp's top-of-hierarchy cgroup pointer (viabpf_cgroup_ancestor(cgrp, 0)) againstroot_cgrp's; skipsinit_cgrp_ctx_with_ancestorsentirely when they differ. Gated behind a new--fix-ignore-cg-v1CLI flag (off by default, plumbed throughrodata.fix_ignore_cg_v1) so the bail remains observable until the operator opts in.Test plan
cargo build --tests --features ktstr-testsbuilds cleanscx_bpf_error (mitosis.bpf.c:149: cgrp_ctx lookup failed for cgid 1)--fix-ignore-cg-v1added to sched_args: passes at ~34 s