Skip to content

Migrate to xunit.v3 #13482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Youssef1313 wants to merge 25 commits into
base: main
Choose a base branch
from
Open

Migrate to xunit.v3 #13482

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
96d7f10
First step to xunit.v3
Youssef1313 Apr 1, 2026
b36d7d3
Remove xunit.abstraction usings
Youssef1313 Apr 1, 2026
ae152fa
Little progress
Youssef1313 Apr 1, 2026
692a543
xunit extensibility
Youssef1313 Apr 1, 2026
86b0608
Progress
Youssef1313 Apr 1, 2026
e264bd3
Adjustments
Youssef1313 Apr 1, 2026
fc99071
Update Directory.Build.targets
Youssef1313 Apr 1, 2026
2ad808e
Coverage
Youssef1313 Apr 1, 2026
8b91ef8
global.json
Youssef1313 Apr 2, 2026
3733ad0
missing conflict resolution
Youssef1313 Apr 2, 2026
1093d4a
Fix before/after test method assertion
Youssef1313 Apr 2, 2026
43bff62
Fix package ref
Youssef1313 Apr 2, 2026
e7b5a6b
Fix for discovery
Youssef1313 Apr 2, 2026
c1d1e24
await, to be more safe
Youssef1313 Apr 2, 2026
71f87ba
Changes from the other PR
Youssef1313 Apr 2, 2026
9770e30
Fix one failure
Youssef1313 Apr 2, 2026
ea6d613
Downgrade verify for now
Youssef1313 Apr 2, 2026
2a83682
Ensure s_runningTests is set early enough before it gets called and c…
Youssef1313 Apr 4, 2026
23628e9
Few fixes for the transition of .NET Framework to be executables
Youssef1313 Apr 4, 2026
411aae6
Additional fixes due to test order changes
Youssef1313 Apr 4, 2026
3b43198
Ensure LogAllEnvironmentVariables state isn't mutated
Youssef1313 Apr 4, 2026
497c085
Stabi
Youssef1313 Apr 5, 2026
1787a85
fix build error
Youssef1313 Apr 8, 2026
15f93d6
Update AspNetCompiler_Tests.cs
Youssef1313 Apr 9, 2026
a8981a3
Update RoslynCodeTaskFactory_Tests.cs
Youssef1313 Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions Coverage.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<?xml version="1.0" encoding="utf-8"?>
<Configuration>
<DeterministicReport>True</DeterministicReport>
<IncludeTestAssembly>true</IncludeTestAssembly>
<ExcludeAssembliesWithoutSources>MissingAll</ExcludeAssembliesWithoutSources>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*doc\.dll$</ModulePath>
<ModulePath>.*customlogger\.dll$</ModulePath>
<ModulePath>.*invalidcustomcheck\.dll$</ModulePath>
<ModulePath>.*portabletask\.dll$</ModulePath>
<ModulePath>.*faultylogger\.dll$</ModulePath>
<ModulePath>.*customcheck\.dll$</ModulePath>
<ModulePath>.*customcheck2\.dll$</ModulePath>
<ModulePath>.*errorcustomcheck\.dll$</ModulePath>
</Exclude>
</ModulePaths>
<Sources>
<Exclude>
<Source>.*\.g\.cs$</Source>
<Source>.*\.sr\.cs$</Source>
</Exclude>
</Sources>
<CommunicationTimeout>120000</CommunicationTimeout>
<PipeClientsCount>1270</PipeClientsCount>
<UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
<EnableStaticNativeInstrumentation>True</EnableStaticNativeInstrumentation>
<EnableDynamicNativeInstrumentation>False</EnableDynamicNativeInstrumentation>
<EnableStaticManagedInstrumentation>False</EnableStaticManagedInstrumentation>
<EnableDynamicManagedInstrumentation>True</EnableDynamicManagedInstrumentation>
</CodeCoverage>
</Configuration>
43 changes: 0 additions & 43 deletions Coverage.runsettings
View file
Open in desktop

This file was deleted.

42 changes: 42 additions & 0 deletions CoverageWindowsFull.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="utf-8"?>
<Configuration>
<DeterministicReport>True</DeterministicReport>
<IncludeTestAssembly>true</IncludeTestAssembly>
<ExcludeAssembliesWithoutSources>MissingAll</ExcludeAssembliesWithoutSources>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*doc\.dll$</ModulePath>
<ModulePath>.*net472.*</ModulePath>
<ModulePath>.*microsoft\.build\.buildcheck\.unittests\.dll$</ModulePath>
<ModulePath>.*microsoft\.build\.tasks\.unittests\.dll$</ModulePath>
</Exclude>
<Include>
<ModulePath>.*microsoft\.build\.engine\.unittests\.dll$</ModulePath>
<ModulePath>.*microsoft\.build\.engine\.om\.unittests\.dll$</ModulePath>
<ModulePath>.*microsoft\.build\.framework\.dll$</ModulePath>
<ModulePath>.*microsoft\.build\.framework\.unittests\.dll$</ModulePath>
<ModulePath>.*microsoft\.build\.commandline\.unittests\.dll$</ModulePath>
<!-- <ModulePath>.*microsoft\.build\.tasks\.core\.dll$</ModulePath> -->
<!-- <ModulePath>.*microsoft\.build\.tasks\.unittests\.shared\.dll$</ModulePath> -->
<!-- <ModulePath>.*microsoft\.build\.utilities\.core\.dll$</ModulePath> -->
<!-- <ModulePath>.*microsoft\.build\.utilities\.unittests\.dll$</ModulePath> -->
<ModulePath>.*microsoft\.net\.stringtools\.dll$</ModulePath>
<ModulePath>.*microsoft\.net\.stringtools\.unittests\.dll$</ModulePath>
</Include>
</ModulePaths>
<Sources>
<Exclude>
<Source>.*\.g\.cs$</Source>
<Source>.*\.sr\.cs$</Source>
</Exclude>
</Sources>
<CommunicationTimeout>120000</CommunicationTimeout>
<PipeClientsCount>1270</PipeClientsCount>
<UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
<EnableStaticNativeInstrumentation>True</EnableStaticNativeInstrumentation>
<EnableDynamicNativeInstrumentation>False</EnableDynamicNativeInstrumentation>
<EnableStaticManagedInstrumentation>False</EnableStaticManagedInstrumentation>
<EnableDynamicManagedInstrumentation>True</EnableDynamicManagedInstrumentation>
</CodeCoverage>
</Configuration>
52 changes: 0 additions & 52 deletions CoverageWindowsFull.runsettings
View file
Open in desktop

This file was deleted.

9 changes: 9 additions & 0 deletions Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,15 @@
</PropertyGroup>

<PropertyGroup>
<TestRunnerName>XUnitV3</TestRunnerName>
<XUnitV3Version>3.2.2</XUnitV3Version>
<MicrosoftTestingPlatformVersion>1.9.1</MicrosoftTestingPlatformVersion>
<OutputType Condition="$(MSBuildProjectName.EndsWith('.UnitTests')) or $(MSBuildProjectName.EndsWith('.Tests'))">Exe</OutputType>

<!-- xUnit1051: Calls to methods which accept CancellationToken should use TestContext.Current.CancellationToken to allow test cancellation to be more responsive. -->
<!-- Consider fixing this later. -->
<NoWarn>$(NoWarn);xUnit1051</NoWarn>

<FullFrameworkTFM>net472</FullFrameworkTFM>

<!--
Expand Down
3 changes: 1 addition & 2 deletions Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<PackageVersion Include="Microsoft.CodeAnalysis.Collections" Version="$(MicrosoftCodeAnalysisCollectionsVersion)" />
<PackageVersion Include="Microsoft.CodeAnalysis.Contracts" Version="$(MicrosoftCodeAnalysisContractsVersion)" />
<PackageVersion Include="Microsoft.CodeAnalysis.PooledObjects" Version="$(MicrosoftCodeAnalysisPooledObjectsVersion)" />
<PackageVersion Include="Microsoft.DotNet.XUnitExtensions" Version="$(MicrosoftDotNetXUnitExtensionsVersion)" />
<PackageVersion Include="Microsoft.DotNet.XUnitV3Extensions" Version="$(MicrosoftDotNetXUnitV3ExtensionsVersion)" />
<PackageVersion Include="Microsoft.IO.Redist" Version="$(MicrosoftIORedistVersion)" />
<PackageVersion Include="Microsoft.Net.Compilers.Toolset" Version="$(MicrosoftNetCompilersToolsetVersion)" Condition="'$(UsingToolMicrosoftNetCompilers)' != 'true'" />
<PackageVersion Include="NuGet.Build.Tasks" Version="$(NuGetBuildTasksVersion)" />
Expand Down Expand Up @@ -46,7 +46,6 @@
<PackageVersion Include="System.Threading.Tasks.Dataflow" Version="$(SystemThreadingTasksDataflowVersion)" />
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="$(SystemThreadingTasksExtensionsVersion)" />
<PackageVersion Include="System.ValueTuple" Version="$(SystemValueTupleVersion)" />
<PackageVersion Include="xunit.console" Version="$(XUnitVersion)" />
<PackageVersion Include="Microsoft.VisualStudio.Telemetry" Version="$(MicrosoftVisualStudioTelemetryVersion)" />
<!-- Microsoft.VisualStudio.SolutionPersistence is maintained in eng/dependabot/Packages.props -->
</ItemGroup>
Expand Down
4 changes: 2 additions & 2 deletions eng/Version.Details.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This file should be imported by eng/Versions.props
<PropertyGroup>
<!-- dotnet-arcade dependencies -->
<MicrosoftDotNetArcadeSdkPackageVersion>10.0.0-beta.26164.1</MicrosoftDotNetArcadeSdkPackageVersion>
<MicrosoftDotNetXUnitExtensionsPackageVersion>10.0.0-beta.26164.1</MicrosoftDotNetXUnitExtensionsPackageVersion>
<MicrosoftDotNetXUnitV3ExtensionsPackageVersion>10.0.0-beta.26164.1</MicrosoftDotNetXUnitV3ExtensionsPackageVersion>
<!-- dotnet-roslyn dependencies -->
<MicrosoftNetCompilersToolsetPackageVersion>5.6.0-2.26151.2</MicrosoftNetCompilersToolsetPackageVersion>
<!-- dotnet-runtime dependencies -->
Expand Down Expand Up @@ -35,7 +35,7 @@ This file should be imported by eng/Versions.props
<PropertyGroup>
<!-- dotnet-arcade dependencies -->
<MicrosoftDotNetArcadeSdkVersion>$(MicrosoftDotNetArcadeSdkPackageVersion)</MicrosoftDotNetArcadeSdkVersion>
<MicrosoftDotNetXUnitExtensionsVersion>$(MicrosoftDotNetXUnitExtensionsPackageVersion)</MicrosoftDotNetXUnitExtensionsVersion>
<MicrosoftDotNetXUnitV3ExtensionsVersion>$(MicrosoftDotNetXUnitV3ExtensionsPackageVersion)</MicrosoftDotNetXUnitV3ExtensionsVersion>
<!-- dotnet-roslyn dependencies -->
<MicrosoftNetCompilersToolsetVersion>$(MicrosoftNetCompilersToolsetPackageVersion)</MicrosoftNetCompilersToolsetVersion>
<!-- dotnet-runtime dependencies -->
Expand Down
2 changes: 1 addition & 1 deletion eng/Version.Details.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>6bdca1d6466c2f9ef46f8e5d3422fbe3568d503c</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.XUnitExtensions" Version="10.0.0-beta.26164.1">
<Dependency Name="Microsoft.DotNet.XUnitV3Extensions" Version="10.0.0-beta.26164.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>e365cfcb58176841c2fb80d1a40926f394776970</Sha>
</Dependency>
Expand Down
7 changes: 5 additions & 2 deletions eng/dependabot/Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,15 +49,18 @@
<PackageVersion Include="Shouldly" Version="4.3.0" />
<PackageVersion Update="Shouldly" Condition="'$(ShouldlyVersion)' != ''" Version="$(ShouldlyVersion)" />

<PackageVersion Include="Microsoft.Testing.Extensions.CodeCoverage" Version="18.0.6" />
<PackageVersion Update="Microsoft.Testing.Extensions.CodeCoverage" Condition="'$(MicrosoftTestingExtensionsCodeCoverageVersion)' != ''" Version="$(MicrosoftTestingExtensionsCodeCoverageVersion)" />

<PackageVersion Include="FakeItEasy" Version="8.1.0" />
<PackageVersion Update="FakeItEasy" Condition="'$(FakeItEasyVersion)' != ''" Version="$(FakeItEasyVersion)" />

<!-- when this is bumped the submodule in https://github.com/dotnet/source-build-reference-packages/tree/main/src/externalPackages needs to be bumped in sync -->
<PackageVersion Include="Microsoft.VisualStudio.SolutionPersistence" Version="1.0.52" />
<PackageVersion Update="Microsoft.VisualStudio.SolutionPersistence" Condition="'$(MicrosoftVisualStudioSolutionPersistenceVersion)' != ''" Version="$(MicrosoftVisualStudioSolutionPersistenceVersion)" />

<PackageVersion Include="Verify.Xunit" Version="19.14.1" />
<PackageVersion Update="Verify.XUnit" Condition="'$(VerifyXUnitVersion)' != ''" Version="$(VerifyXUnitVersion)" />
<PackageVersion Include="Verify.XunitV3" Version="28.12.0" />
<PackageVersion Update="Verify.XunitV3" Condition="'$(VerifyXunitV3Version)' != ''" Version="$(VerifyXunitV3Version)" />
</ItemGroup>

<!-- Roslyn analyzer authoring packages (used by ThreadSafeTaskAnalyzer) -->
Expand Down
3 changes: 3 additions & 0 deletions global.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
],
"errorMessage": "The .NET SDK could not be found, please run a command-line build with ./build.cmd."
},
"test": {
"runner": "Microsoft.Testing.Platform"
},
"tools": {
"dotnet": "10.0.104",
"vs": {
Expand Down
1 change: 0 additions & 1 deletion src/Build.OM.UnitTests/Construction/ProjectFormatting_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@
using Microsoft.Build.Shared;
using Microsoft.Build.UnitTests;
using Xunit;
using Xunit.Abstractions;

#nullable disable

Expand Down
1 change: 0 additions & 1 deletion src/Build.OM.UnitTests/Construction/ProjectMetadataElement_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
using Microsoft.Build.Construction;
using Microsoft.Build.Evaluation;
using Xunit;
using Xunit.Abstractions;
using InvalidProjectFileException = Microsoft.Build.Exceptions.InvalidProjectFileException;

#nullable disable
Expand Down
1 change: 0 additions & 1 deletion src/Build.OM.UnitTests/Definition/ProjectCollection_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
using Microsoft.Build.Utilities;
using Shouldly;
using Xunit;
using Xunit.Abstractions;
using InvalidProjectFileException = Microsoft.Build.Exceptions.InvalidProjectFileException;

#nullable disable
Expand Down
1 change: 0 additions & 1 deletion src/Build.OM.UnitTests/Definition/Project_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
using Microsoft.Build.Shared;
using Shouldly;
using Xunit;
using Xunit.Abstractions;
using GlobResultList = System.Collections.Generic.List<(string, string[], System.Collections.Immutable.ImmutableHashSet<string>, System.Collections.Immutable.ImmutableHashSet<string>)>;
using InvalidProjectFileException = Microsoft.Build.Exceptions.InvalidProjectFileException;
// can't use an actual ProvenanceResult because it points to a ProjectItemElement which is hard to mock.
Expand Down
1 change: 0 additions & 1 deletion src/Build.OM.UnitTests/Instance/ProjectInstance_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
using Microsoft.Build.Framework;
using Microsoft.Build.Shared;
using Xunit;
using Xunit.Abstractions;
using ForwardingLoggerRecord = Microsoft.Build.Logging.ForwardingLoggerRecord;

#nullable disable
Expand Down
1 change: 0 additions & 1 deletion src/Build.OM.UnitTests/NugetRestoreTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
using Microsoft.Build.UnitTests.Shared;
using Shouldly;
using Xunit;
using Xunit.Abstractions;

namespace Microsoft.Build.Engine.OM.UnitTests
{
Expand Down
7 changes: 6 additions & 1 deletion src/Build.UnitTests/BackEnd/AppHostSupport_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
using Microsoft.Build.UnitTests;
using Shouldly;
using Xunit;
using Xunit.Abstractions;

#nullable disable

Expand Down Expand Up @@ -92,6 +91,12 @@ public void ClearBootstrapDotnetRootEnvironment_ClearsVariablesNotInOriginalEnvi
env.SetEnvironmentVariable("DOTNET_ROOT", @"C:\TestDotnet");
env.SetEnvironmentVariable("DOTNET_ROOT_X64", @"C:\TestDotnetX64");

// This might seem redundant, but it's not.
// ClearBootstrapDotnetRootEnvironment is a production method that alters env variables directly via Environment class.
// This clears DOTNET_ROOT_X86 which might have been originally set.
// To ensure that DOTNET_ROOT_X86 is restored, we set it to itself so that TestEnvironment.Dispose restores it back.
env.SetEnvironmentVariable("DOTNET_ROOT_X86", Environment.GetEnvironmentVariable("DOTNET_ROOT_X86"));

// Original environment does NOT have these variables
var originalEnvironment = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);

Expand Down
1 change: 0 additions & 1 deletion src/Build.UnitTests/BackEnd/AssemblyTaskFactory_E2E_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
using Microsoft.Build.UnitTests.Shared;
using Shouldly;
using Xunit;
using Xunit.Abstractions;

namespace Microsoft.Build.Engine.UnitTests
{
Expand Down
1 change: 0 additions & 1 deletion src/Build.UnitTests/BackEnd/BuildManager_Logging_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
using Microsoft.Build.UnitTests;
using Shouldly;
using Xunit;
using Xunit.Abstractions;
using static Microsoft.Build.UnitTests.ObjectModelHelpers;

#nullable disable
Expand Down
1 change: 0 additions & 1 deletion src/Build.UnitTests/BackEnd/BuildManager_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
using Microsoft.Build.Utilities;
using Shouldly;
using Xunit;
using Xunit.Abstractions;
using static Microsoft.Build.UnitTests.ObjectModelHelpers;
using Constants = Microsoft.Build.Framework.Constants;

Expand Down
1 change: 0 additions & 1 deletion src/Build.UnitTests/BackEnd/BuildRequestConfiguration_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
using Microsoft.Build.Shared;
using Shouldly;
using Xunit;
using Xunit.Abstractions;

#nullable disable

Expand Down
3 changes: 2 additions & 1 deletion src/Build.UnitTests/BackEnd/DebugUtils_tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ public class DebugUtils_Tests
public void DumpExceptionToFileShouldWriteInDebugDumpPath()
{
DebugUtils.ResetDebugDumpPathInRunningTests = true;
var exceptionFilesBefore = Directory.GetFiles(DebugUtils.DebugDumpPath, "MSBuild_*failure.txt");
var exceptionFilesBefore =
Directory.Exists(DebugUtils.DebugDumpPath) ? Directory.GetFiles(DebugUtils.DebugDumpPath, "MSBuild_*failure.txt") : Array.Empty<string>();
Comment on lines +23 to +24
Copy link
Copy Markdown
Member Author

@Youssef1313 Youssef1313 Apr 7, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some unknown reason, sometimes the DebugDumpPath directory isn't there on disk, and then GetFiles throws an exception if the directory doesn't exist.

Maybe there is some other previous test that deletes DebugDumpPath or something. Anyways, it should be safe to keep this change. It just means that future refactoring for how tests work is ideal to avoid this kind of cross-test dependency issues.


string[] exceptionFiles = null;

Expand Down
1 change: 0 additions & 1 deletion src/Build.UnitTests/BackEnd/LoggingContext_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
using Microsoft.Build.Framework;
using Shouldly;
using Xunit;
using Xunit.Abstractions;

#nullable disable

Expand Down
1 change: 0 additions & 1 deletion src/Build.UnitTests/BackEnd/MSBuild_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
using Microsoft.Build.Utilities;

using Xunit;
using Xunit.Abstractions;

#nullable disable

Expand Down
Loading