test: add GOROOT runner baseline on main#1750
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the testing infrastructure by introducing a GOROOT-based test runner. This new system allows for direct validation of Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a new GOROOT test runner, allowing llgo to be tested against upstream Go GOROOT/test cases across various Go toolchains. It includes a shell script (dev/test_goroot.sh) to orchestrate multi-GOROOT test runs, a Go test package (test/goroot) for discovering, executing, and comparing test outputs, and platform-specific process management for robust child process handling. The review suggests improving the log timestamp parsing in runner_test.go for better readability and resilience, and using more descriptive reasons for xfail entries in xfail.yaml to enhance long-term maintainability.
|
Well-structured test infrastructure. The runner design (symlinked work trees, process group management, xfail/timeout config, sharding) is solid and the unit test coverage is good. Key items to address:
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1750 +/- ##
==========================================
- Coverage 93.16% 92.69% -0.47%
==========================================
Files 48 49 +1
Lines 13349 13397 +48
==========================================
- Hits 12437 12419 -18
- Misses 726 781 +55
- Partials 186 197 +11 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
6042f69 to
e434045
Compare
Summary
// runrunner and baseline xfails on top of the latestmainxfail/host_skipentries without changing compiler or runtime implementation1.24.x/1.25.x/1.26.xacross macOS and Ubuntu in2shards1.25as the required gating version; keep Go1.24and Go1.26as observational coverage for nowScope
979selectedGOROOT/test// runcases in the current runner configurationGOROOT/testtreeCurrent XFail Accounting
Notes
Runner Scope Baselineis the selected runner subset size used as a reporting baseline; actual discovered case counts can vary byGOVERSION/GOOS/GOARCHafter build-constraint filteringdarwin/arm64also keeps5host_skipcases for known host-unsafe concurrency testslinux/amd64currently uses only version-specificxfailentries