AP_Compass: report specific failure reason when fit rejected

[christianpetri]

Summary

Extend the CompassCalibrator::Status enum with three new terminal failure values (FAILED_OFFSETS=8, FAILED_DIAG_SCALING=9, FAILED_RESIDUALS_HIGH=10) and refactor fit_acceptable() into a pure-query fit_acceptable_status() const so that MAG_CAL_REPORT.cal_status reports the specific reason calibration was rejected instead of always returning the generic FAILED.

Classification & Testing (check all that apply and add your own)

• Checked by a human programmer
• Automated test(s) verify changes (e.g. unit test, autotest)
• Tested manually, description below (e.g. SITL)
• Tested on hardware (MatekF405-Wing V2 / ArduPlane)

Description

Previously fit_acceptable() returned a silent false — the GCS and user had no way to know why calibration was rejected. This PR maps each rejection condition to a specific Status value that is transmitted in the MAG_CAL_REPORT.cal_status field.

New Status enum values (extend MAG_CAL_STATUS MAVLink enum):

Value	Name	Condition
8	FAILED_OFFSETS	Any offset component ≥ COMPASS_OFFS_MAX
9	FAILED_DIAG_SCALING	Diagonal or off-diagonal scaling out of range
10	FAILED_RESIDUALS_HIGH	_fitness > sq(_tolerance) or NaN

FAILED_RADIUS (7) was already present. All four conditions were previously collapsed into FAILED (5).

Behaviour changes:

• fit_acceptable() replaced by fit_acceptable_status() const — pure query, no side effects; returns the specific Status directly.
• All call sites explicitly call set_status() with the returned value.
• set_status(FAILED) is guarded against overwriting any of the five specific FAILED_* statuses (previously only FAILED_ORIENTATION and FAILED_RADIUS were protected).
• Retry-hold timer (COMPASS_CAL_RETRY_REPORT_HOLD_MS = 500 ms) ensures a GCS polling MAG_CAL_REPORT at a few Hz reliably observes the specific failure reason before the next retry overwrites it.
• update_cal_status(), failed(), is_calibrating(), and send_mag_cal_report() all handle the three new cases identically to FAILED_RADIUS.
• Flash cost +64 bytes on MatekF405-Wing plane (measured locally).

Note: The new integer values (8/9/10) are transmitted immediately. GCS will display the raw number until a matching MAG_CAL_STATUS entry is added in the upstream mavlink/mavlink XML and a pymavlink bump lands in ArduPilot. The autotest uses getattr(mavutil.mavlink, 'MAG_CAL_FAILED_RESIDUALS_HIGH', 10) fallbacks pending that bump.

Related

• Discussion: https://discuss.ardupilot.org/t/proposal-early-abort-compass-calibration-when-magnetic-field-is-extreme/143374
• MAVLink enum update: common: rename/add MAG_CAL_STATUS failure enums to FAILED_* + RESIDUALS_HIGH mavlink/mavlink#2478 — adds MAG_CAL_FAILED_OFFSETS/FAILED_DIAG_SCALING/FAILED_RESIDUALS_HIGH to common.xml
• MissionPlanner GUI update: ConfigHWCompass: surface MAG_CAL_FAILED_OFFSETS / FAILED_DIAG_SCALING / FAILED_RESIDUALS_HIGH failure codes MissionPlanner#3722 — surfaces failure codes in calibration dialogs

Testing

Builds clean on SITL and MatekF405-Wing (./waf configure --board MatekF405-Wing && ./waf plane). Flash delta vs master: +64 bytes on MatekF405-Wing plane.

SITL autotest SITLCompassCalibration passes:

Tools/autotest/autotest.py test.Copter.SITLCompassCalibration

Two subtests verify the new codes:

1. Tight-fit test (COMPASS_CAL_FIT=0.001) — all compasses report MAG_CAL_FAILED_RESIDUALS_HIGH (10) instead of MAG_CAL_FAILED (5).
2. Selective-failure test (SIM_MAG1_OFS_X/Y/Z=2000) — one compass reports MAG_CAL_FAILED_OFFSETS (8); remaining compasses still succeed.

Hardware test on MatekF405-Wing V2 with ArduPlane: successful calibration completes normally. With a strong nearby magnet (field ~1500 mGauss), FAILED_RADIUS is reported in Mission Planner messages, confirming the specific status is visible instead of the previous silent generic failure.

AI-assisted contribution: This PR was developed with GitHub Copilot (Claude Sonnet 4.6). The human author has reviewed every line, run the tests, and is fully responsible for the submitted code.

[IamPete1]

I think 950 is too low, that is within the range of offsets that we might see in normal operation. I think we might be better off to try and improve the reporting at the end of a calibration attempt rather than guessing at the beginning.

[rmackay9]

Hi @christianpetri,

Thanks for this. One small thing is we don't accept merge commits so could you rebase on master to remove it?

[peterbarker]

Create an autotest

• want other compasses to keep calibrating if one of them gets a long field
• we have lots of autotests already doing calibration, steal code from there
• does this make calibration faster with a badly-affected compass?

Generally in favour of this at DevCallEU

What's the flash cost?

[IamPete1]

Not keen on the short circuit stuff, but I like the improved reporting at the end of the calibration. To get the most out of this we might need some GCS changes, in mission planner you can't see messages on the compass calibration page, you would have to go and look manualy.

[christianpetri]

Dropped the early-abort in pull_sample() as requested. Kept and improved the fit_acceptable() diagnostics: each failure condition (fitness NaN, radius out of range, offsets too large, diagonal/offdiagonal out of range, fitness over tolerance) now emits its own GCS message with the actual values, so all failures are visible in a single calibration attempt rather than just the first hit. Removed const from fit_acceptable() since GCS_SEND_TEXT is a side effect. This is consistent with the pre-existing messages already in calculate_orientation() and fix_radius().

[christianpetri]

SITL Test Plan

Manual SITL verification for the Mag(N) bad ... STATUSTEXT capture in the compass calibration dialog.

• Mission Planner now shows the Mag(N) bad fit diagnostics directly in the compass calibration panel, so the failure reason is visible immediately without leaving the page.

• Close-up of the Onboard Mag Calibration box showing a specific rejection reason (offsets over limit) alongside the live calibration progress.

Related PRs

• ArduPilot: AP_Compass: report specific failure reason when fit rejected #32757 (emits Mag(N) bad ... diagnostics)
• Mission Planner: ConfigHWCompass: show calibration bad-fit reasons in mag cal dialog MissionPlanner#3706 (captures and displays them)

Setup

sim_vehicle.py -v ArduCopter -f calibration --console --map

Connect Mission Planner via TCP 127.0.0.1:5762, navigate to Initial Setup → Mandatory Hardware → Compass.

Reset between tests

param set COMPASS_CAL_FIT 16
param set SIM_MAG1_OFS_X 0 ; param set SIM_MAG1_OFS_Y 0 ; param set SIM_MAG1_OFS_Z 0
param set SIM_MAG2_OFS_X 0 ; param set SIM_MAG2_OFS_Y 0 ; param set SIM_MAG2_OFS_Z 0
param set SIM_MAG3_OFS_X 0 ; param set SIM_MAG3_OFS_Y 0 ; param set SIM_MAG3_OFS_Z 0
sitl_stop ; sitl_attitude 0 0 0

Tests

#	Scenario	Setup	Action	Expected
1	Success baseline	(defaults)	Start → sitl_magcal → Accept	Progress/report lines, no bad lines
2	Bad fit	COMPASS_CAL_FIT 0.001	Start → sitl_magcal	Mag(N) bad fit: fitness X tolerance Y per compass
3	Bad offset (early abort)	SIM_MAG1_OFS_X/Y/Z 2000	Start → sitl_magcal	Mag(0) bad fit: ofs (2000,2000,2000)>=1800
4	Cancel/restart	(defaults)	Start → Cancel mid-run → Start	Cancel/restart of calibration via MP — diagnostics state resets cleanly (verified with #3706)

Pass criteria

• Success path shows no bad lines.
• Failure path shows Mag(N) bad ... lines, line-separated, latest-per-compass.
• Restart is clean after Cancel.
• Non-compass STATUSTEXT still flows to HUD/messages.
• No UI freeze or crash.

[IamPete1]

I like this! And Thanks for doing the Mission Planner fix too!

[christianpetri]

Physical Hardware Test Results

Tested on MatekF405-Wing with Mission Planner (using ArduPilot/MissionPlanner#3722 - MAVLink bump that includes new compass cal status values).

Firmware: commit 94b0ca3 (compass-cal-early-abort branch)
Flash cost: +64 bytes on MatekF405-Wing plane (measured)
Failure Mode Put a magnet close to the compass (Status: magfield). Note: Magnet should not be too close otherwise it gets stuck at approx. 33%.

The PR now correctly reports specific failure reasons instead of generic "Compass calibration failed":

Baseline Test: Success

Test 1: Radius Failure (Attempt 2, 74% complete)

Message: "Compass calibration failed: the radius of the fitted sphere is unrealistically small or large"

Test 2: Offset Magnitude Failure (Attempt 2, 12% complete)

Message: "Compass calibration failed: offset magnitude too large. [x:408.8 y:1988.0 z:-1747.3 limit ±1800]"

---

Notes

• With latest Mission Planner PR (with MAVLink bump from mavlink/mavlink#2478): Shows descriptive failure messages as seen above
• With older Mission Planner (no MAVLink bump): Shows raw MAVLink cal_status enum value (e.g., status=8 for MAG_CAL_FAILED_OFFSETS, status=9 for MAG_CAL_FAILED_DIAG_SCALING, status=10 for MAG_CAL_FAILED_RESIDUALS_HIGH)

The specific failure reporting works as designed! Users can now understand why calibration failed and take appropriate corrective action.

[christianpetri]

Additional Notes: Progress Stall Detection

During testing, I observed a scenario where calibration progress can stall indefinitely:

Scenario: Magnetic Field Too Strong

When the magnetic field is extremely strong (magnet very close to compass), the calibration can get stuck at approximately 33% progress (during step two sample collection). The field distortion is so severe that the algorithm cannot collect valid samples across the required orientations, causing the progress bar to freeze.

Current Behavior

• The PR correctly reports specific failure reasons when the fit is rejected after sample collection completes
• However, if sample collection never completes (stalls), the calibration remains in the running state indefinitely
• No timeout or stall detection is implemented

Attempted Solution

I explored adding a timeout/stall detection mechanism to abort calibration when progress stops advancing. The challenge is:

• The calibration state machine uses AP_HAL::millis() for timing
• SITL autotest uses simulated time, which makes timeout testing complex
• Need to ensure timeout logic works correctly in both real hardware and SITL environments

Future Work

A stall detection mechanism would be a valuable addition:

• Detect when no new samples are collected for N seconds
• Report FAILED_SAMPLE_TIMEOUT or similar status
• Requires careful design to work with both real time (hardware) and sim time (SITL)

I decided to defer this work to keep the current PR focused on the core improvement (specific failure reporting when fit validation fails). The stall detection feature can be addressed in a separate PR with proper SITL testing infrastructure.

---

Note: Users can manually cancel stalled calibrations via Mission Planner/GCS. The stall scenario requires unusually strong magnetic interference (magnet within a few cm / mm of compass).

[christianpetri]

Hi @hamishwillee and @peterbarker, is there anything that I should add before it can be reviewed or is it ready?

[rmackay9]

we should definitely try and get this one in so I've added it (again) to the weekly dev call

[christianpetri]

Note on further refactoring: fix_radius() and calculate_orientation() still return bool and call set_status() internally - the same pattern that motivated the fit_acceptable_status() refactor. Extending them to return Status directly would remove the case Status::FAILED guard in set_status() and the dead fallback in update(), but that touches more lines than this PR's core scope. Happy to include it here if preferred, or defer to a follow-up.

[peterbarker]

@christianpetri This is going to fail in CI due to the bad commit message.

[tridge]

Automated review note — AI-generated (Claude), validated against the live diff. Please sanity-check before acting.

Review verdict: COMMENT

Genuinely useful diagnostics (users currently get an opaque "calibration failed"), correctly wired to the MAVLink enum. The retry-hold's effect on buffer lifetime is the main thing to think about.

Findings

• ISSUE — The retry path now holds the sample buffer for the report-hold window. Previously the FAILED_* case did if (_retry && set_status(WAITING_TO_START)) { _attempt++; return true; }, which reset state immediately. Now it sets _retry_pending and returns, deferring set_status(WAITING_TO_START) to update() up to 500 ms later. The buffer is reused rather than leaked (RUNNING_STEP_ONE only allocates if (_sample_buffer == nullptr)), so this is a hold not a leak — but it is COMPASS_CAL_RETRY_REPORT_HOLD_MS of extra RAM pressure per calibrating compass on every failed attempt, which matters on 1 MB boards calibrating three compasses at once. CompassCalibrator.cpp:500
• NOTE — The CI failure is unrelated infrastructure flake — verified. build-test fails in ardupilot_dds_tests on test_plane_airspeed.py::test_dds_udp_airspeed_msg_recv_plane, with micro_ros_agent dying on a segfault (exit code -11). Nothing to do with compass calibration; a re-run should clear it.
• NOTE — The new enum values match upstream MAVLink — verified. FAILED_OFFSETS = 8, FAILED_DIAG_SCALING = 9 and FAILED_RESIDUALS_HIGH = 10 are exactly the values in MAG_CAL_STATUS in ArduPilot/mavlink, and ArduPilot master's pinned modules/mavlink (13f2f735) already contains them — so neither a submodule bump nor a pymavlink update is needed for the autotest's mavutil.mavlink.MAG_CAL_FAILED_* references. CompassCalibrator.h:46
• NOTE — isnan(_fitness) mapped to FAILED_RESIDUALS_HIGH is semantically off. A NaN fitness is a numerical breakdown, not "residuals too high"; reporting the generic FAILED for it would be more honest and would keep the new specific codes meaningful to a GCS. CompassCalibrator.cpp:521
• NOTE — The _running() switch is correct. Changing update()'s guard from running() to _running() is necessary, not cosmetic: update() already holds state_sem via WITH_SEMAPHORE, and the public running() takes the same non-recursive semaphore again. _running() is the pre-existing lock-free variant (CompassCalibrator.cpp:345).
• NOTE — The comparison inversions in fit_acceptable_status() are exactly equivalent to the old conjunction — I checked each of the radius, offset, diagonal and off-diagonal bounds. No behavioural change beyond the reason reporting.

---

Generated from an automated review of all PRs labelled DevCallTopic on 2026-07-28. Full report: https://uav.tridgell.net/DevCallReviews/DevCallTopic/devcall_pr_reviews.html

[peterbarker]

@christianpetri if you want to come back with a PR for the NaN thing, please feel free!

Thanks for this contribution. I know it's been a lot of round-trips for you.