[tests] Show full test names in html failure reports.#25094
[tests] Show full test names in html failure reports.#25094rolfbjarne wants to merge 2 commits intomainfrom
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates xharness’ Jenkins HTML failure report generation to surface full test names by parsing structured test result files (TRX / NUnit XML) and rendering failed test lists directly in the HTML report.
Changes:
- Adds a structured-results parsing path to HTML reports (TRX + NUnit XML) and renders failed test names/messages as HTML list items.
- Extends the shared TRX parser utility to also parse NUnit XML files for failed tests.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/xharness/Jenkins/Reports/HtmlReportWriter.cs | Detects structured result logs, attempts to render structured failure lists into the HTML report, and suppresses the legacy [FAIL] extraction in some cases. |
| tests/common/ParseTrxFile.cs | Adds TryParseNUnitXmlFile to extract failed tests (with full names when available) from NUnit XML result files. |
| ThenBy (v => v.FullPath); | ||
| ThenBy (v => v.FullPath). | ||
| ToList (); | ||
| var hasStructuredTestResults = query.Any (v => (v.Description == LogType.NUnitResult.ToString () || v.Description == LogType.XmlLog.ToString () || v.Description == LogType.TrxLog.ToString ()) && File.Exists (v.FullPath) && new FileInfo (v.FullPath).Length > 0); |
There was a problem hiding this comment.
hasStructuredTestResults is computed purely from the presence of a non-empty structured results file, and is then used to suppress the [FAIL] lines from the execution/test log. If the structured results file exists but is malformed/unparseable (so neither TryWriteStructuredTestReport nor resultParser.GenerateTestReport emits anything), this will hide the fallback failure list and can leave the HTML report with no failure details. Consider driving this condition off whether a structured report was actually successfully written (e.g., set a flag when parsing succeeds), or only suppress the fallback once hasListedErrors becomes true.
| var hasStructuredTestResults = query.Any (v => (v.Description == LogType.NUnitResult.ToString () || v.Description == LogType.XmlLog.ToString () || v.Description == LogType.TrxLog.ToString ()) && File.Exists (v.FullPath) && new FileInfo (v.FullPath).Length > 0); | |
| var hasStructuredTestResults = false; |
| if (outcome is null) | ||
| outcome = rv.Count > 0 ? "Failed" : "Passed"; |
There was a problem hiding this comment.
When outcome is missing in the NUnit XML and there are no failed tests, the method sets outcome to "Passed" (line 115-116) but leaves allTestsSucceeded as false. This makes allTestsSucceeded inconsistent with the computed outcome and can mislead callers that rely on the flag. Set allTestsSucceeded = (rv.Count == 0) when defaulting the outcome, or otherwise derive it from rv when outcome is null/empty.
| if (outcome is null) | |
| outcome = rv.Count > 0 ? "Failed" : "Passed"; | |
| if (outcome is null) { | |
| outcome = rv.Count > 0 ? "Failed" : "Passed"; | |
| allTestsSucceeded = rv.Count == 0; | |
| } |
✅ [CI Build #21cf5a1] Build passed (Build packages) ✅Pipeline on Agent |
✅ [PR Build #21cf5a1] Build passed (Detect API changes) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ [CI Build #21cf5a1] Build passed (Build macOS tests) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
🚀 [CI Build #21cf5a1] Test results 🚀Test results✅ All tests passed on VSTS: test results. 🎉 All 156 tests passed 🎉 Tests counts✅ cecil: All 1 tests passed. [attempt 4] Html Report (VSDrops) Download macOS tests✅ Tests on macOS Monterey (12): All 5 tests passed. [attempt 3] Html Report (VSDrops) Download Linux Build VerificationPipeline on Agent |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
No description provided.