Skip to content

coverlet.console: add trace diagnostics and actionable warnings for instrumentation/hit/empty-result failures - #2005

Merged
Bertk merged 11 commits into
coverlet-coverage:masterfrom
Bertk:improve-console-error-messages
Aug 1, 2026
Merged

coverlet.console: add trace diagnostics and actionable warnings for instrumentation/hit/empty-result failures#2005
Bertk merged 11 commits into
coverlet-coverage:masterfrom
Bertk:improve-console-error-messages

Conversation

@Bertk

@Bertk Bertk commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR improves troubleshooting in coverlet.console when coverage output is empty or unexpectedly 0% (especially in CI-only repros).
It adds:

• Trace verbosity support.
• --diag option to write detailed diagnostics to file.
• Better warning/error reporting across:

  1. instrumentation eligibility/failures,
  2. no-hit scenarios,
  3. empty coverage result scenarios.

Problem

coverlet.console can complete successfully while producing an empty coverage report (no modules) or all-zero results, with limited diagnostics.
In CI, this is hard to root-cause because skipped instrumentation and missing-hit conditions are not summarized clearly.

Changes

  1. coverlet.console trace diagnostics
    • Extended LogLevel with Trace.
    • Updated CLI verbosity help to include trace.
    • Added --diag option in Program:
    • writes timestamped diagnostics to file,
    • useful for CI artifact collection,
    • does not break normal console behavior.

  2. Console logger enhancements
    • Added diagnostic file sink support (EnableDiagnosticFile).
    • Added trace logging method usage path (file + console depending on level).
    • Preserved existing thread-safe console output behavior.

  3. Instrumentation diagnostics improvements (coverlet.core)
    • Added reason-aware CanInstrument(out string reason) overload in Instrumenter.
    • Coverage.PrepareModules() now logs explicit skip reasons and aggregates:
    • skipped not eligible,
    • skipped preflight,
    • instrumentation exceptions.
    • Added warning when selected modules exist but none are instrumented.

  4. Hit collection and empty-result diagnostics
    • Coverage.CalculateCoverage() now tracks modules with non-zero hits.
    • Added warning when instrumented modules exist but no hits are collected.
    • Coverage.GetCoverageResult() now warns when final CoverageResult.Modules is empty and includes context counts.

  5. Documentation update
    • Updated Documentation/GlobalTool.md:
    • Trace verbosity listed in options.
    • --diag option documented.
    • Added troubleshooting guidance for:
    • instrumentation failures,
    • no-hit runs,
    • empty coverage result output.

Why this helps

With this PR, CI logs (and optional diag artifacts) provide direct guidance instead of silent/ambiguous 0% output.
This reduces time-to-diagnose for environment-specific instrumentation and coverage collection issues.

Backward Compatibility

• Default behavior remains unchanged.
• Existing command lines continue to work.
• New diagnostics are additive.
• No breaking exit-code changes introduced.

Example usage

coverlet "<test_bin_dir>" `
  --target "dotnet" `
  --targetargs "<test_dll_path>" `
  --output "<results>\coverage.coverlet-console.json" `
  --verbosity trace `
  --diag "<results>\coverlet.console.trace.log"

Added --diag <file> option to the coverlet CLI and global tool to capture all trace-level diagnostics in a file, aiding CI troubleshooting. Introduced a new Trace log level and updated --verbosity to accept trace. Enhanced ConsoleLogger to support diagnostic file output. Improved engine diagnostics for skipped modules, missing hits, and empty results, with actionable log messages. Refactored CanInstrument for better skip reasons. Updated docs and test scripts; bumped .NET SDK to 10.0.302.
@Bertk Bertk added enhancement General enhancement request driver-console Issue related to dotnet net tool driver coverlet-core labels Aug 1, 2026
dependabot Bot and others added 7 commits August 1, 2026 12:48
Bumps [actions/stale](https://github.com/actions/stale) from 10 to 11.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](actions/stale@v10...v11)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: '11'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…rlet-coverage#1998)

Bumps [marocchino/sticky-pull-request-comment](https://github.com/marocchino/sticky-pull-request-comment) from 3.0.4 to 3.0.5.
- [Release notes](https://github.com/marocchino/sticky-pull-request-comment/releases)
- [Commits](marocchino/sticky-pull-request-comment@v3.0.4...v3.0.5)

---
updated-dependencies:
- dependency-name: marocchino/sticky-pull-request-comment
  dependency-version: 3.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-dotnet](https://github.com/actions/setup-dotnet) from 5 to 6.
- [Release notes](https://github.com/actions/setup-dotnet/releases)
- [Commits](actions/setup-dotnet@v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-dotnet
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…overlet-coverage#1996)

Bumps [danielpalme/ReportGenerator-GitHub-Action](https://github.com/danielpalme/reportgenerator-github-action) from 5.5.10 to 5.5.11.
- [Release notes](https://github.com/danielpalme/reportgenerator-github-action/releases)
- [Commits](danielpalme/ReportGenerator-GitHub-Action@5.5.10...5.5.11)

---
updated-dependencies:
- dependency-name: danielpalme/ReportGenerator-GitHub-Action
  dependency-version: 5.5.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4 to 4.37.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@v4...v4.37.3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.37.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…t-coverage#1999)

---
updated-dependencies:
- dependency-name: dotnet-reportgenerator-globaltool
  dependency-version: 5.5.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Changed log level for messages about skipping non-instrumentable modules from LogWarning to LogVerbose. This reduces log noise by making these messages less prominent in standard logs.
@Bertk
Bertk marked this pull request as ready for review August 1, 2026 11:38
Copilot AI review requested due to automatic review settings August 1, 2026 11:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request enhances Coverlet’s troubleshooting experience when coverlet.console produces empty/0% coverage by adding trace verbosity, an optional diagnostic log file sink, and more actionable warnings in the instrumentation and hit-collection pipeline.

Changes:

  • Added Trace verbosity and a --diag <path> option to persist detailed diagnostics to a file from coverlet.console.
  • Improved instrumentation diagnostics in coverlet.core by logging skip reasons, aggregating instrumentation outcomes, and warning on “no instrumented modules” / “no hits” / “empty result modules”.
  • Updated CI/test scripts, tool/workflow versions, and global tool documentation to reflect the new diagnostics capabilities.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/coverlet.core/Instrumentation/Instrumenter.cs Adds CanInstrument(out reason) for reason-aware eligibility checks and improved diagnostics.
src/coverlet.core/Coverage.cs Aggregates instrumentation outcomes; warns on no hits and empty module results.
src/coverlet.console/Program.cs Adds --diag CLI option and wires it into the console logger.
src/coverlet.console/Logging/LogLevel.cs Introduces Trace log level.
src/coverlet.console/Logging/ConsoleLogger.cs Adds diagnostic file sink and a trace logging path.
scripts/test.ps1 Updates test runner progress option usage.
global.json Bumps .NET SDK version.
Documentation/GlobalTool.md Documents trace verbosity and --diag, adds troubleshooting guidance.
.github/workflows/issue-label-close.yml Bumps actions/stale version.
.github/workflows/dotnet.yml Bumps action versions used in CI and ReportGenerator/sticky-comment steps.
.github/workflows/codeql.yml Pins CodeQL action versions and bumps setup-dotnet action.
.github/workflows/benchmark.yml Bumps setup-dotnet action.
.config/dotnet-tools.json Bumps reportgenerator tool version.

Comment thread src/coverlet.console/Logging/ConsoleLogger.cs
Comment thread src/coverlet.console/Logging/ConsoleLogger.cs
Comment thread src/coverlet.core/Instrumentation/Instrumenter.cs
Bertk and others added 3 commits August 1, 2026 13:48
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@Bertk
Bertk merged commit 4032f53 into coverlet-coverage:master Aug 1, 2026
10 checks passed
@Bertk
Bertk deleted the improve-console-error-messages branch August 1, 2026 14:18
Bertk added a commit that referenced this pull request Aug 2, 2026
* Improve log formatting for multi-line messages in console

- Update changelog for `coverlet.console` to note new trace diagnostics and actionable warnings for instrumentation, hit, and empty-result failures (PR #2005).
- Refine `ConsoleLogger.cs` so that when logging multi-line messages (e.g., coverage tables), the timestamp and log level header are written on a separate line from the message body, ensuring clean rendering of tables and summaries in log files.

* use platform newline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coverlet-core driver-console Issue related to dotnet net tool driver enhancement General enhancement request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

coverlet.console: Improve diagnostics for silent instrumentation/collection failures (add trace + actionable warnings)

2 participants