Skip to content

test: add GOROOT runner baseline on main#1750

Open
cpunion wants to merge 44 commits intogoplus:mainfrom
cpunion:xfail/goroot-main-20260326
Open

test: add GOROOT runner baseline on main#1750
cpunion wants to merge 44 commits intogoplus:mainfrom
cpunion:xfail/goroot-main-20260326

Conversation

@cpunion
Copy link
Copy Markdown
Collaborator

@cpunion cpunion commented Mar 26, 2026

Summary

  • add the GOROOT // run runner and baseline xfails on top of the latest main
  • record current GOROOT failures as xfail / host_skip entries without changing compiler or runtime implementation
  • add a dedicated GitHub Actions workflow that runs the GOROOT runner on Go 1.24.x / 1.25.x / 1.26.x across macOS and Ubuntu in 2 shards
  • keep Go 1.25 as the required gating version; keep Go 1.24 and Go 1.26 as observational coverage for now

Scope

  • current runner baseline scope is 979 selected GOROOT/test // run cases in the current runner configuration
  • this is the current selected runner subset, not the whole GOROOT/test tree

Current XFail Accounting

Go Platform Runner Scope Baseline xfail host_skip Expected Normal Pass
1.24 darwin/arm64 979 134 5 840
1.24 linux/amd64 979 133 0 846
1.25 darwin/arm64 979 165 5 809
1.25 linux/amd64 979 151 0 828
1.26 darwin/arm64 979 440 5 534
1.26 linux/amd64 979 495 0 484

Notes

  • Runner Scope Baseline is the selected runner subset size used as a reporting baseline; actual discovered case counts can vary by GOVERSION / GOOS / GOARCH after build-constraint filtering
  • darwin/arm64 also keeps 5 host_skip cases for known host-unsafe concurrency tests
  • linux/amd64 currently uses only version-specific xfail entries
  • follow-up PRs should fix compiler/runtime causes; this PR only establishes runner coverage and tracks current failures

@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the testing infrastructure by introducing a GOROOT-based test runner. This new system allows for direct validation of llgo against a comprehensive suite of upstream Go tests, improving compatibility and catching regressions early. It includes mechanisms for handling platform-specific test behaviors and integrates into the continuous integration pipeline to provide ongoing feedback on llgo's performance against standard Go behavior.

Highlights

  • GOROOT Runner Integration: A new GOROOT // run runner has been added to execute upstream Go tests against llgo without copying source files.
  • Baseline XFails and Host Skips: Initial baseline xfails and host-unsafe skips for darwin/arm64 failures have been recorded to reflect current main branch status.
  • Dedicated GitHub Actions Workflow: A new GitHub Actions workflow has been introduced to run the GOROOT runner on Go 1.25 across macOS and Ubuntu, sharded into 4 parts.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/goroot.yml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new GOROOT test runner, allowing llgo to be tested against upstream Go GOROOT/test cases across various Go toolchains. It includes a shell script (dev/test_goroot.sh) to orchestrate multi-GOROOT test runs, a Go test package (test/goroot) for discovering, executing, and comparing test outputs, and platform-specific process management for robust child process handling. The review suggests improving the log timestamp parsing in runner_test.go for better readability and resilience, and using more descriptive reasons for xfail entries in xfail.yaml to enhance long-term maintainability.

@xgopilot
Copy link
Copy Markdown
Contributor

xgopilot bot commented Mar 26, 2026

Well-structured test infrastructure. The runner design (symlinked work trees, process group management, xfail/timeout config, sharding) is solid and the unit test coverage is good.

Key items to address:

  1. go test default timeout — the shell script doesn't pass -timeout, so go test will kill the runner at 10 minutes (see inline comment on test_goroot.sh).
  2. Version prefix matching bug"go1.2" would incorrectly match "go1.24".
  3. CI cost — 8 matrix jobs on every push to every branch is expensive; consider scoping triggers.
  4. Missing Linux xfails — all entries are darwin/arm64 only; ubuntu shards may fail.

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 26, 2026

Codecov Report

❌ Patch coverage is 54.76190% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.69%. Comparing base (2075aa9) to head (0187bbc).
⚠️ Report is 22 commits behind head on main.

Files with missing lines Patch % Lines
internal/plan9asm/translate.go 0.00% 9 Missing and 1 partial ⚠️
internal/env/env.go 62.50% 5 Missing and 1 partial ⚠️
test/goroot/proc_unix.go 66.66% 1 Missing and 1 partial ⚠️
cl/instr.go 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1750      +/-   ##
==========================================
- Coverage   93.16%   92.69%   -0.47%     
==========================================
  Files          48       49       +1     
  Lines       13349    13397      +48     
==========================================
- Hits        12437    12419      -18     
- Misses        726      781      +55     
- Partials      186      197      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cpunion cpunion force-pushed the xfail/goroot-main-20260326 branch from 6042f69 to e434045 Compare March 31, 2026 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant