Skip to content

Fix Linux XDP Debug BVT runner crash by adding swap space#5919

Draft
ProjectsByJackHe wants to merge 2 commits intomainfrom
jackhe/add-swap-space
Draft

Fix Linux XDP Debug BVT runner crash by adding swap space#5919
ProjectsByJackHe wants to merge 2 commits intomainfrom
jackhe/add-swap-space

Conversation

@ProjectsByJackHe
Copy link
Copy Markdown
Contributor

@ProjectsByJackHe ProjectsByJackHe commented Apr 6, 2026

Description

The BVT (Debug, linux, ubuntu-24.04, x64, quictls, -UseSystemOpenSSLCrypto, -Test, -UseXdp) job consistently
crashes the GitHub Actions runner at ~72 minutes — test step logs return HTTP 404 and cleanup steps never run.
The Release variant of the same matrix passes reliably.

Root cause: OOM. GitHub-hosted runners have 7 GB RAM. The Debug + XDP combination exhausts this due to:

  • Unoptimized Debug binaries (larger memory footprint)
  • XDP UMEM allocations (16K frames of pinned kernel memory per AF_XDP socket)
  • Parallel test execution under sudo

When the Linux OOM-killer fires, it can kill the runner agent itself (not just the test process), causing the
full runner disconnect.

This adds a 6 GB swap file on Linux XDP runners before tests run, giving ~13 GB total virtual memory. The swap
file is on the runner's ephemeral SSD so performance impact is minimal. Only Linux XDP matrices are affected —
all other jobs are unchanged.

Testing

CI

Documentation

No

@ProjectsByJackHe ProjectsByJackHe requested a review from a team as a code owner April 6, 2026 21:27
@ProjectsByJackHe ProjectsByJackHe marked this pull request as draft April 6, 2026 21:37
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.80%. Comparing base (f9c01c7) to head (b39cc8e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5919      +/-   ##
==========================================
- Coverage   86.18%   85.80%   -0.38%     
==========================================
  Files          60       60              
  Lines       18731    18731              
==========================================
- Hits        16143    16073      -70     
- Misses       2588     2658      +70     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant