Skip to content

feat: Compare generated code against reference files for tests. Provide metrics for some aspects of the generated code - #2348

Merged
kevinferrare merged 1 commit into
masterfrom
feature/generated_code_measures_in_tests
Sep 16, 2026
Merged

kevinferrare merged 1 commit into
masterfrom
feature/generated_code_measures_in_tests

Conversation

@kevinferrare

Copy link
Copy Markdown
Contributor

Preliminary phase for code generator improvements

Copilot AI lite review requested due to automatic review settings September 15, 2026 22:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One or more issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds generated-code golden snapshots and Roslyn-based metrics to strengthen code-generator regression testing.

Changes:

  • Compares generated C# against committed reference files.
  • Exposes compilation artifacts and records syntax/diagnostic metrics.
  • Documents golden regeneration.
File summaries
File Description
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/strings.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/sticli.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_smc_guard.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_mixed_block.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_invalid_opcode.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_discard.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_convergence.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_closure.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_call_entry.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/speculative_branch.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/selfmodifyvalue.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/selfmodifyterminator.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/selfmodifyje.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/selfmodifyinstructions.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/selfmodifycalltarget.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/selfmodifycall.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/segpr.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/returnedterminator.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/pushcs_callnear_retfar.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_shared_tail.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_mutual_recursion_unwind.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_multi_entry_irreducible_shared.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_multi_entry_dominated_shared.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_mixed_activation_cycle.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_jump_into_function_middle.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_indirect_call_jump.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/partition_cross_function_loop.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/multimisalignedcall.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/lockprefix.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/linearsamesegmenteddifferent.a20.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/jump2.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/jmpmov.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/interrupt.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/intchain.ivt.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/externalint.pit.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/divfaultloop.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/div2.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/datatrnf.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/control.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/Resources/cpuTests/res/GeneratedCode/cmpneg.spec.cs.txt Updated as part of this pull request.
tests/Spice86.Tests/GeneratedOverrideCompiler.cs Updated as part of this pull request.
tests/Spice86.Tests/GeneratedCompilation.cs Updated as part of this pull request.
tests/Spice86.Tests/GeneratedCodeMetricsCollector.cs Updated as part of this pull request.
tests/Spice86.Tests/GeneratedCodeMetrics.cs Updated as part of this pull request.
tests/Spice86.Tests/GeneratedCodeMachineTestRunner.cs Updated as part of this pull request.
AGENTS.md Updated as part of this pull request.
Review details

Suppressed comments (2)

tests/Spice86.Tests/GeneratedCodeMachineTestRunner.cs:65

  • MaxCycles and ConfigureMachine are omitted from this key even though both are passed into GenerateProgram and can change the discovered CFG and generated source. Two tests using the same binary and flags with different values will therefore share one golden and can validate against the wrong scenario; include every source-affecting input in the key or require callers to provide a distinct key.
    /// overwriting each other. <c>MaxCycles</c> and <c>ConfigureMachine</c> are deliberately not part
    /// of the key. Every flag is enumerated explicitly so adding one to the options is a
    /// compile-visible decision here rather than a silent omission that lets two runs share a golden.

tests/Spice86.Tests/GeneratedCodeMachineTestRunner.cs:124

  • This new helper adds an optional/defaulted parameter, contrary to the repository's no-optional-parameters guidance. Keep the regeneration-only path explicit instead of introducing a callable-without-a-path API here.
    private static string GetSourceFilePath([CallerFilePath] string path = "") => path;
  • Files reviewed: 56/57 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/Spice86.Tests/GeneratedCodeMachineTestRunner.cs
Comment thread tests/Spice86.Tests/GeneratedCodeMetricsCollector.cs
…de metrics for some aspects of the generated code
@kevinferrare
kevinferrare force-pushed the feature/generated_code_measures_in_tests branch from cf3ad3b to b8a6365 Compare September 16, 2026 06:32
@kevinferrare
kevinferrare merged commit 2b599b9 into master Sep 16, 2026
5 checks passed
@kevinferrare
kevinferrare deleted the feature/generated_code_measures_in_tests branch September 16, 2026 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants