Skip to content

Test every shipping stripping level - #516

Merged
Eli Pinkerton (wallstop) merged 3 commits into
masterfrom
perf/session-257-shipping-stripping-matrix
Sep 1, 2026
Merged

Test every shipping stripping level#516
Eli Pinkerton (wallstop) merged 3 commits into
masterfrom
perf/session-257-shipping-stripping-matrix

Conversation

@wallstop

@wallstop Eli Pinkerton (wallstop) commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Why

Shipping-fidelity CI only tested High managed stripping. It could not catch level-specific AOT root failures in production-shaped players.

What changed

  • Add reviewed profiles for Minimal, Low, and Medium stripping.
  • Keep the existing High profile identity.
  • Generate each player's exact stripping level from its validated profile.
  • Build isolated players on the oldest and newest supported Unity editors.
  • Run positive and missing-root checks twice from each binary.
  • Keep engine stripping disabled until the shipping builder loads.
  • Preserve later profile evidence after a caught profile failure.
  • Update workflow guards, tests, runbooks, and agent guidance.

How we know

  • The script suite passes 518 tests.
  • Repository validation, formatting, spelling, actionlint, and yamllint pass.
  • The Unity MCP EditMode assembly passes 817 tests.
  • The editor remains idle with one clean main-stage scene after testing.
  • Final-head CI has 37 successful checks and 5 intentional skips.
  • All four Unity jobs pass.
  • Both endpoints pass every stripping level and both runtime modes.
  • Both performance jobs pass.
  • Bugbot reports no final-head issues.
  • All review threads are resolved.

Remaining scope

This advances #506 without closing it. Larger message topologies and durable native evidence remain separate work.

Refs #506

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit acf60d7. Configure here.

Comment thread .github/workflows/unity-tests.yml Outdated
Comment thread .github/workflows/unity-tests.yml Outdated
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Performance Numbers

Measured commit: 49c54a4
Trusted reporting code (PR base): af246b5
Workflow run: 33483815309 attempt 1

Benchmark or harness paths changed, so historical deltas are non-comparable:

  • scripts/unity/run-ci-tests.ps1
  • scripts/unity/validate-il2cpp-profile.ps1

Historical Standalone delta

Delta direction: + is better; - is worse.

Historical delta omitted because benchmark or harness code changed in this pull request.

Current TargetMap evidence

These raw rows are current-run diagnostics. Use fresh bracketed controls before accepting a micro-optimization.

[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_16_Churn keyCount=16 operation=Churn totalOperations=2520000 operationsPerSecond=502439.103 wallClockMs=5015.533 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=16 targetMapCapacity=32 observedInvocations=2540000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_16_Hit keyCount=16 operation=Hit totalOperations=76290000 operationsPerSecond=15256611.038 wallClockMs=5000.455 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=16 targetMapCapacity=32 observedInvocations=76310000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_16_Miss keyCount=16 operation=Miss totalOperations=81490000 operationsPerSecond=16296921.796 wallClockMs=5000.331 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=16 targetMapCapacity=32 observedInvocations=0
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_1_Churn keyCount=1 operation=Churn totalOperations=2610000 operationsPerSecond=521464.060 wallClockMs=5005.139 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=1 targetMapCapacity=4 observedInvocations=2630000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_1_Hit keyCount=1 operation=Hit totalOperations=75150000 operationsPerSecond=15028648.924 wallClockMs=5000.450 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=1 targetMapCapacity=4 observedInvocations=75170000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_1_Miss keyCount=1 operation=Miss totalOperations=80830000 operationsPerSecond=16165445.202 wallClockMs=5000.172 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=1 targetMapCapacity=4 observedInvocations=0
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_256_Churn keyCount=256 operation=Churn totalOperations=2420000 operationsPerSecond=481895.879 wallClockMs=5021.832 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=256 targetMapCapacity=512 observedInvocations=2440000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_256_Hit keyCount=256 operation=Hit totalOperations=71320000 operationsPerSecond=14262674.427 wallClockMs=5000.465 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=256 targetMapCapacity=512 observedInvocations=71340000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_256_Miss keyCount=256 operation=Miss totalOperations=79770000 operationsPerSecond=15953834.399 wallClockMs=5000.052 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=256 targetMapCapacity=512 observedInvocations=0
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_4096_Churn keyCount=4096 operation=Churn totalOperations=2130000 operationsPerSecond=425023.127 wallClockMs=5011.492 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=4096 targetMapCapacity=8192 observedInvocations=2150000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_4096_Hit keyCount=4096 operation=Hit totalOperations=61420000 operationsPerSecond=12282582.836 wallClockMs=5000.577 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=4096 targetMapCapacity=8192 observedInvocations=61440000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_4096_Miss keyCount=4096 operation=Miss totalOperations=77400000 operationsPerSecond=15478188.123 wallClockMs=5000.585 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=4096 targetMapCapacity=8192 observedInvocations=0
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_4_Churn keyCount=4 operation=Churn totalOperations=2520000 operationsPerSecond=503973.279 wallClockMs=5000.265 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=4 targetMapCapacity=8 observedInvocations=2540000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_4_Hit keyCount=4 operation=Hit totalOperations=75640000 operationsPerSecond=15127988.503 wallClockMs=5000.004 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=4 targetMapCapacity=8 observedInvocations=75660000
[Standalone] DXM_TARGET_MAP_BENCHMARK scenario=TargetMap_4_Miss keyCount=4 operation=Miss totalOperations=82210000 operationsPerSecond=16441770.473 wallClockMs=5000.070 gcAllocations=-1 gcAllocatedBytes=-1 targetMapEntries=4 targetMapCapacity=8 observedInvocations=0
[Standalone] DXM_TARGET_MAP_CONSTRUCTION keyCount=1 wallClockMs=0.000102 operationsPerSecond=9832841.691 gcAllocations=-1 gcAllocatedBytes=-1 targetMapCapacity=4
[Standalone] DXM_TARGET_MAP_CONSTRUCTION keyCount=16 wallClockMs=0.000625 operationsPerSecond=1600000.000 gcAllocations=-1 gcAllocatedBytes=-1 targetMapCapacity=32
[Standalone] DXM_TARGET_MAP_CONSTRUCTION keyCount=256 wallClockMs=0.007750 operationsPerSecond=129032.258 gcAllocations=-1 gcAllocatedBytes=-1 targetMapCapacity=512
[Standalone] DXM_TARGET_MAP_CONSTRUCTION keyCount=4 wallClockMs=0.000210 operationsPerSecond=4770992.366 gcAllocations=-1 gcAllocatedBytes=-1 targetMapCapacity=8
[Standalone] DXM_TARGET_MAP_CONSTRUCTION keyCount=4096 wallClockMs=0.124800 operationsPerSecond=8012.821 gcAllocations=-1 gcAllocatedBytes=-1 targetMapCapacity=8192

@wallstop
Eli Pinkerton (wallstop) merged commit 2eb11c1 into master Sep 1, 2026
42 checks passed
@wallstop
Eli Pinkerton (wallstop) deleted the perf/session-257-shipping-stripping-matrix branch September 1, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant