Skip to content

fix(analysis): analyzer validation checks source project success instead of test project success - #3391

Closed
rouke-broersma with Copilot wants to merge 2 commits into
masterfrom
copilot/fix-analyzer-results-issue
Closed

fix(analysis): analyzer validation checks source project success instead of test project success#3391
rouke-broersma with Copilot wants to merge 2 commits into
masterfrom
copilot/fix-analyzer-results-issue

Conversation

Copilot AI commented Jan 6, 2026

Copy link
Copy Markdown
Contributor

The analyzer validation incorrectly rejected projects for mutation when test projects failed analysis, even though the source project succeeded.

Root Cause

findMutableAnalyzerResults is a Dictionary<IAnalyzerResult, List<IAnalyzerResult>> where:

  • Key: source/mutable project to be mutated
  • Value: test projects that reference it

The validation at line 112 checked if test projects succeeded:

if (findMutableAnalyzerResults.All(p => p.Value.All(r => !r.Succeeded)))

This conflicted with the error message which correctly reported the source projects as failed.

Changes

  • InputFileResolver.cs:112: Changed validation to check source project success: !p.Key.Succeeded
  • Tests: Added coverage for both scenarios:
    • Source succeeds + test fails → proceeds with mutation
    • Source fails → throws InputException

The fix allows mutation testing when test project analysis fails but source project analysis succeeds, which is the common case for many valid projects.

Original prompt

This section details on the original issue you should resolve

<issue_title>Analyzer results are invalid if test project analyzer result is not succeeded</issue_title>
<issue_description>Describe the bug

if (findMutableAnalyzerResults.All(p => p.Value.All(r => !r.Succeeded)) )
considers a project invalid for mutation if it's mapped test projects analyzer results are invalid, I'm not sure why this is the case but while working on #3386 I am observing that this is the case for many projects that otherwise mutation test without issues after changing the condition to check the target project succeeded instead.

@dupdob can you clarify whether this is a bug or if this check is required for another scenario?
</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…st projects

The previous implementation incorrectly checked if test projects succeeded instead of checking if the mutable/target projects succeeded. This caused valid projects to be rejected when test projects failed to analyze but the source project succeeded.

Changed the condition at line 112 from checking p.Value (test projects) to checking p.Key (mutable projects).

Added two tests:
- ShouldSucceedWhenTestProjectFailsButSourceProjectSucceeds: Verifies that when test project fails but source succeeds, no exception is thrown
- ShouldThrowWhenSourceProjectFailsAnalysis: Verifies that when source project fails, an exception is thrown

Co-authored-by: rouke-broersma <3595932+rouke-broersma@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix invalid analyzer results handling for mutation tests Fix analyzer validation to check source project success instead of test project success Jan 6, 2026
Copilot AI requested a review from rouke-broersma January 6, 2026 07:25
@rouke-broersma
rouke-broersma marked this pull request as ready for review January 6, 2026 07:28
Copilot AI review requested due to automatic review settings January 6, 2026 07:28
@rouke-broersma rouke-broersma changed the title Fix analyzer validation to check source project success instead of test project success fix(analysis): analyzer validation checks source project success instead of test project success Jan 6, 2026
@sonarqubecloud

sonarqubecloud Bot commented Jan 6, 2026

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a critical logic error in analyzer validation that incorrectly rejected source projects for mutation when test project analysis failed, even though the source projects succeeded. The fix changes the validation to check source project success status instead of test project success status, which aligns with the error message that was already reporting failed source projects.

Key Changes

  • Changed the validation condition from checking test project success to checking source project success
  • Updated test name to better reflect the new behavior
  • Added comprehensive test coverage for both scenarios (test failure with source success, and source failure)

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
InputFileResolver.cs Fixed validation logic at line 112 to check !p.Key.Succeeded (source project) instead of p.Value.All(r => !r.Succeeded) (test projects)
InputFileResolverTests.cs Renamed test from ShouldHandleFailedAnalysis to ShouldSucceedWhenTestProjectFailsButSourceProjectSucceeds and added new test ShouldThrowWhenSourceProjectFailsAnalysis for comprehensive coverage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Analyzer results are invalid if test project analyzer result is not succeeded

3 participants