Skip to content

ch_tests_tool: parse nextest JUnit output for cloud-hypervisor tests#4495

Open
anirudhrb wants to merge 1 commit into
microsoft:mainfrom
anirudhrb:ch_integration_nextest
Open

ch_tests_tool: parse nextest JUnit output for cloud-hypervisor tests#4495
anirudhrb wants to merge 1 commit into
microsoft:mainfrom
anirudhrb:ch_integration_nextest

Conversation

@anirudhrb
Copy link
Copy Markdown
Collaborator

Description

The cloud-hypervisor integration suite now emits per-suite JUnit XML via cargo nextest. Prefer that structured output over the legacy cargo stdout scraping so subtest results are accurate and resilient to log-format churn upstream.

  • Collect JUnit XMLs from the SSH user's $HOME/workloads/junit/, which is where dev_cli.sh's docker container (run with --user $(id -u):$(id -g) and bind-mounting $HOME/workloads) writes them as the SSH user.
  • Resolve $HOME on the remote node before each use, falling back to /root/workloads/junit if resolution fails.
  • Wipe any stale JUnit dir from a previous run on the same node before starting tests so results can never be misattributed across runs.
  • Parse elements into CloudHypervisorTestResult, mapping failure/error -> FAILED, skipped -> SKIPPED, otherwise PASSED. Subtests expected but missing from every JUnit file stay QUEUED. Last retry wins, matching nextest's final-outcome reporting.
  • When 'cargo nextest' appears in stdout but no JUnit files are produced, fail with a clear diagnostic instead of silently falling back to the stdout parser.
  • Fall back to the existing stdout parser when no JUnit files are present and nextest wasn't invoked (classic cargo test path).

Related Issue

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring
  • Documentation update

Checklist

  • Description is filled in above
  • No credentials, secrets, or internal details are included
  • Peer review requested (if not, add required peer reviewers after raising PR)
  • Tests executed and results posted below

Test Validation

Key Test Cases:

Impacted LISA Features:

Tested Azure Marketplace Images:

Test Results

Image VM Size Result
PASSED / FAILED / SKIPPED

The cloud-hypervisor integration suite now emits per-suite JUnit XML via
cargo nextest. Prefer that structured output over the legacy cargo stdout
scraping so subtest results are accurate and resilient to log-format
churn upstream.

- Collect JUnit XMLs from the SSH user's $HOME/workloads/junit/, which
  is where dev_cli.sh's docker container (run with --user $(id -u):$(id
  -g) and bind-mounting $HOME/workloads) writes them as the SSH user.
- Resolve $HOME on the remote node before each use, falling back to
  /root/workloads/junit if resolution fails.
- Wipe any stale JUnit dir from a previous run on the same node before
  starting tests so results can never be misattributed across runs.
- Parse <testcase> elements into CloudHypervisorTestResult, mapping
  failure/error -> FAILED, skipped -> SKIPPED, otherwise PASSED. Subtests
  expected but missing from every JUnit file stay QUEUED. Last retry
  wins, matching nextest's final-outcome reporting.
- When 'cargo nextest' appears in stdout but no JUnit files are
  produced, fail with a clear diagnostic instead of silently falling
  back to the stdout parser.
- Fall back to the existing stdout parser when no JUnit files are
  present and nextest wasn't invoked (classic cargo test path).

Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Copilot AI review requested due to automatic review settings May 25, 2026 14:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Cloud Hypervisor test runner tool to prefer parsing cargo nextest JUnit XML output (copied back from the VM) over the legacy stdout scraping, improving subtest result fidelity and resilience to upstream log-format changes.

Changes:

  • Collect nextest-generated JUnit XML files from the remote node and parse <testcase> elements into CloudHypervisorTestResult statuses.
  • Add a guardrail to fail with a clear diagnostic when nextest appears to have run but no JUnit output is found.
  • Proactively remove any stale remote JUnit output directory before running tests to avoid misattributing results across runs.

Comment thread lisa/microsoft/testsuites/cloud_hypervisor/ch_tests_tool.py
Comment thread lisa/microsoft/testsuites/cloud_hypervisor/ch_tests_tool.py
Comment thread lisa/microsoft/testsuites/cloud_hypervisor/ch_tests_tool.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants