Skip to content

Add 354 unit tests and calibrate mutation testing thresholds toward 80% - #95

Open
Burgyn with Copilot wants to merge 11 commits into
masterfrom
copilot/add-mutation-testing
Open

Add 354 unit tests and calibrate mutation testing thresholds toward 80%#95
Burgyn with Copilot wants to merge 11 commits into
masterfrom
copilot/add-mutation-testing

Conversation

Copilot AI commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Baseline mutation score is 37.83% — low by industry standards (80%+ is good). This PR adds comprehensive test coverage for previously untested code and recalibrates Stryker thresholds to drive improvement.

Threshold changes

Updated stryker-config.json from break=35/low=40/high=60 to:

"thresholds": {
  "high": 80,
  "low": 50,
  "break": 37
}
  • break=37: At baseline — any regression fails CI immediately
  • low=50: Mid-term target
  • high=80: Industry-standard good score

Documentation

Added honest score assessment to docs/mutation-testing.md with industry benchmark ranges and root cause analysis (596 no-coverage + 707 survived mutants in console/pipeline/IO code).

New tests (354 tests across 48 files)

Coverage added for previously untested subsystems:

  • Utilities & validatorsPascalCaseHumanizer, StringExtensions, Timer, Constants, ExitCode, VariableNameValidator, FunctionNameValidator, HeaderNameValidator
  • HTTP parsingMethodAndUriParser, HeaderParser, BodyParser, CommentLineParser, EmptyLineParser, HttpParsingContext
  • Header handlers — All 10 implementations (Default, Authorization, ContentType, Host, UserAgent, Date, Connection, ContentDisposition, ContentEncoding, ContentLanguage)
  • Body resolversJsonBodyResolver, XmlBodyResolver
  • Builder patternsApplicationContextOptionsBuilder, BaseDirectivePatternBuilder, HttpDirectivePatternBuilder
  • Models & recordsTestResultsSummary, CollectionTestResultsSummary, File, Folder, InternalFile, ExternalFile, Script, TestCase, ScriptReference, TestDescription, RequestVariableDescription
  • Registries & structureExternalFilesRegistry, CollectionStructure, NoAuthProvider
  • Directives & constantsAuthConstants, AuthDirectives, RetryingConstants, RetryingDirectives, TestDirectives
  • Path resolutionPathProvider, RelativePathResolver, PathExtensions
  • ContextsRequestExecutionContext, ApplicationContextOptions, HttpParsingContext
  • Retry parsersRetryStrategyDirectiveLineParser, RetryUntilStatusCodesDirectiveLineParser, RetryExplicitPropertiesDirectiveLineParser
  • Functions & JSONFunction<T> variants, DefaultFunctionsRegistrator, CaseInsensitiveExpandoObject, Environment

Test count: 449 → 803 passing. The 1 pre-existing failure (AuthHttpMessageHandlerShould) is unchanged.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.deps.json /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/testhost.dll --port 34997 --endpoint 127.0.0.1:034997 --role client --parentprocessid 4326 --telemetryoptedin false (dns block)
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.deps.json /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/testhost.dll --port 33035 --endpoint 127.0.0.1:033035 --role client --parentprocessid 5388 --telemetryoptedin false (dns block)
    • Triggering command: /usr/share/dotnet/dotnet /usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.runtimeconfig.json --depsfile /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/TeaPie.Tests.deps.json /home/REDACTED/work/TeaPie/TeaPie/tests/TeaPie.Tests/bin/Debug/net8.0/testhost.dll --port 39445 --endpoint 127.0.0.1:039445 --role client --parentprocessid 7310 --telemetryoptedin false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.

Copilot AI changed the title [WIP] Introduce mutation testing to improve test suite confidence Add mutation testing with Stryker.NET Mar 26, 2026
Copilot AI requested a review from Burgyn March 26, 2026 20:43
Break threshold raised to 37 (at baseline - no regression allowed),
low set to 50 (mid-term target), high set to 80 (industry good).
Documentation updated with honest assessment and score ranges.

Agent-Logs-Url: https://github.com/Kros-sk/TeaPie/sessions/aa8c45ed-90bf-4506-b48f-74d67cd27895

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
Copilot AI changed the title Add mutation testing with Stryker.NET Add Stryker.NET mutation testing with honest score assessment Mar 27, 2026
Copilot AI and others added 6 commits March 27, 2026 05:51
Add test files following existing conventions ({ClassName}Should.cs pattern,
xUnit with FluentAssertions, file-scoped namespaces):

- PascalCaseHumanizerShould: SplitPascalCase with various inputs
- StringExtensionsShould: TrimSuffix edge cases
- ConstantsShould: Verify all constant values
- VariableNameValidatorShould: Valid/invalid variable name validation
- FunctionNameValidatorShould: Valid/invalid function name validation
- HeaderNameValidatorShould: HTTP header name/value validation
- JsonBodyResolverShould: JSON body resolution with JPath
- XmlBodyResolverShould: XML body resolution with XPath
- RequestVariableDescriptionShould: Record behavior
- TimerShould: All 3 Execute overloads
- ExitCodeShould: Enum value verification
- VariableNameViolationExceptionShould: Exception constructors
- FunctionNameViolationExceptionShould: Exception constructors

Total: 83 new passing tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
…ion types, and ScriptReference

Add 15 new test files with 77 tests covering:
- HTTP parsing: MethodAndUriParser, HeaderParser, BodyParser, CommentLineParser, EmptyLineParser
- Testing summaries: TestResultsSummary, CollectionTestResultsSummary, TestCaseTestResultsSummary
- Structure exploration: File, Folder, InternalFile, ExternalFile, Script, TestCase
- Scripts: ScriptReference

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
…pattern builders

- ApplicationContextOptionsBuilderShould: 8 tests covering defaults, setters, fluent chaining, null handling
- ApplicationContextOptionsShould: 5 tests covering constructors, null defaults, property mutability
- BaseDirectivePatternBuilderShould: 8 tests covering pattern building, params, prefix, auto-naming
- HttpDirectivePatternBuilderShould: 4 tests covering factory, prefix, params, sample directive matching

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
Add test files for DefaultHeaderHandler, AuthorizationHeaderHandler,
HostHeaderHandler, UserAgentHeaderHandler, DateHeaderHandler,
ConnectionHeaderHandler, ContentTypeHeaderHandler,
ContentEncodingHeaderHandler, ContentLanguageHeaderHandler, and
ContentDispositionHeaderHandler.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
…d Testing

Add 8 test files with 54 tests covering:
- ExternalFilesRegistry: register, get, overwrite, unregistered key
- CollectionStructure: constructors, folders, test cases, env file, init script
- NoAuthProvider: authenticate completes without modifying request
- AuthConstants: constant value verification
- AuthDirectives: prefix, names, regex pattern matching
- RetryingConstants: default values and backoff type
- RetryingDirectives: full names and regex pattern matching
- TestDirectives: prefix, full names, and regex pattern matching

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
…nments (batch 4: 77 tests)

Agent-Logs-Url: https://github.com/Kros-sk/TeaPie/sessions/fac89cfe-d408-4f06-a17d-75ebe01fbef8

Co-authored-by: Burgyn <5930822+Burgyn@users.noreply.github.com>
Copilot AI changed the title Add Stryker.NET mutation testing with honest score assessment Add 354 unit tests and calibrate mutation testing thresholds toward 80% Mar 27, 2026
@Burgyn
Burgyn marked this pull request as ready for review March 31, 2026 19:09
@Burgyn

Burgyn commented Apr 1, 2026

Copy link
Copy Markdown
Member

Code review

Found 1 issue:

  1. ContainSingle("value1") uses the wrong FluentAssertions overload -- the string parameter is treated as a "because" failure message, not the expected value. This assertion only verifies the collection has exactly one element but does not verify the element is "value1". The test passes regardless of what value the header contains. Fix: use .ContainSingle().Which.Should().Be("value1") or .ContainSingle(x => x == "value1").

_handler.SetHeader("value1", request);
request.Headers.GetValues("X-Custom").Should().ContainSingle("value1");
}

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

…rHandlerShould

ContainSingle(string) treats the argument as a failure reason message, not the
expected value. Use ContainSingle().Which.Should().Be() to actually assert the value.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants