Speed up slowest acceptance tests and add --no-build consistently#15471
Open
nohwnd wants to merge 3 commits intomicrosoft:mainfrom
Open
Speed up slowest acceptance tests and add --no-build consistently#15471nohwnd wants to merge 3 commits intomicrosoft:mainfrom
nohwnd wants to merge 3 commits intomicrosoft:mainfrom
Conversation
- Reduce HangDumpChildProcesses TestTimeout from 15s to 10s (saves ~5s per parameterization). The child processes hang immediately, so 10s is plenty. - Add --no-build to second InvokeDotnetTest call in RunSettingsAreLoadedFromProject to skip redundant rebuild when only runsettings change. - Standardize --no-build placement right after the path argument in all InvokeDotnetTest calls for consistency. - Remove unused using directive in DotnetTestMSBuildOutputTests.cs. - Add trx-analysis skill for parsing TRX test result files. Fixes microsoft#15468 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes the slowest acceptance tests by reducing timeouts and adding --no-build flags to avoid redundant builds, while also standardizing argument ordering and removing an unused using directive.
Changes:
- Reduced
HangDumpChildProcessesTestTimeoutfrom 15s to 10s to save ~10s total. - Added
--no-buildtoInvokeDotnetTestcalls across multiple test files and removedPackageVersionMSBuild property from those calls. - Added a new
.github/skills/trx-analysis/SKILL.mdwith PowerShell TRX analysis queries.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
RunsettingsTests.cs |
Adds --no-build to both InvokeDotnetTest calls and removes PackageVersion property |
DotnetTestTests.cs |
Adds --no-build to all InvokeDotnetTest calls and removes PackageVersion property |
DotnetTestMSBuildOutputTests.cs |
Adds --no-build, removes PackageVersion property and unused using |
BlameDataCollectorTests.cs |
Reduces hang dump TestTimeout from 15s to 10s |
.github/skills/trx-analysis/SKILL.md |
New skill file with reusable PowerShell TRX analysis queries |
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DotnetTestTests.cs
Outdated
Show resolved
Hide resolved
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DotnetTestMSBuildOutputTests.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses #15468 — analyzed the slowest acceptance tests from a net48_x86 TRX run and applied targeted optimizations.
Changes
Performance optimizations
TestTimeoutfrom 15s to 10s. The child processes hang immediately viaThread.Sleep(30_000), so 10s is ample margin (the simplerHangDumpOnTimeouttest already uses 3s). Saves ~5s per parameterization (~10s total).--no-buildto the secondInvokeDotnetTestcall. The project is already built by the first call; the second only changes runsettings (a test-execution concern, not a build concern). Saves ~10-15s per parameterization (~20-30s total).Consistency
--no-buildplacement to always be right after the path argument in allInvokeDotnetTestcalls acrossDotnetTestTests.cs,DotnetTestMSBuildOutputTests.cs, andRunsettingsTests.cs.using Microsoft.TestPlatform.TestUtilitiesfromDotnetTestMSBuildOutputTests.cs.Tooling
trx-analysisskill (.github/skills/trx-analysis/SKILL.md) with reusable PowerShell queries for parsing TRX files: slowest tests, most-executed tests, failure analysis, etc.Top 10 slowest tests (from TRX analysis)
Estimated total savings: ~30-40s across affected parameterizations.
Also filed #15470 for the broader issue of over-parameterized tests (e.g.
RunTestsFromMultipleMSTestAssembliesruns 140 times).