Skip to content

Debug-build test failures masked by Eigen assertions (mac-coverage job) #2546

Description

@dcwhite

Summary

The mac-coverage CI job (the only job in the matrix built with -DCMAKE_BUILD_TYPE=Debug; see reusable-build.yml and scripts/coverage.sh) has 32/799 tests failing, all pre-existing on master (confirmed via master's direct-push run 28214300264, not introduced by any recent branch).

These failures don't show up as red in CI because the ctest/coverage steps in reusable-build.yml all have continue-on-error: true, so the job reports success regardless.

Root cause

Release builds define NDEBUG, which strips out Eigen's eigen_assert() (a thin wrapper over assert()). Debug builds don't, so mac-coverage is the only job where these asserts can actually fire. That means the underlying issues (invalid Eigen calls) exist in all build configs — Release builds just silently proceed into undefined/wrong behavior instead of crashing.

Failures to investigate

  1. Core_Datatypes_TestsDyadicTensorTest.DifferentDimensionsNotEquivalent

    Assertion failed: (dimensions_match(m_leftImpl.dimensions(), m_rightImpl.dimensions())), function evalSubExprsIfNeeded, file TensorAssign.h, line 146.
    

    Test intentionally compares tensors of different dimensions; something in the comparison path performs an Eigen Tensor op that asserts instead of returning false/handling the mismatch gracefully.

  2. .Test.ExampleNetwork.negative_tensors_srn and .Test.ExampleNetwork.negative_tensors_normals_deubg_srn (regression networks)

    Assertion failed: (rows() == cols()), function inverse, file InverseImpl.h, line 351.
    

    Some tensor-field module is calling .inverse() on a non-square matrix while processing this network's data.

  3. Algorithms_DataIO_TestsWriteMatrixTest.CanPrintSparseMatrix (assertion-free, a genuine output mismatch)

    Expected: "Nonzero entries:\n(1,0) (_,_) (-1.4,2) ...\nOuter pointers:\n0 2 4  $..."
    Actual:   "Nonzero entries:\n(1,0) (-1.4,2) (_,_) ...\nOuter pointers:\n0 1 6  $..."
    

    Sparse-matrix entry ordering in the debug-print differs from what the test expects — likely stale expected output rather than a real bug, but needs a look.

Suggested next steps

  • Reproduce locally with a Debug build (./build.sh --debug -DBUILD_TESTING=ON) and run the three failing suites/tests individually.
  • For Save network file from GUI #1 and Load network file into GUI #2, find the DyadicTensor / tensor-field code path(s) invoking .inverse() or tensor comparison ops and add a guard (or fix the underlying non-square/mismatched-dimension case) instead of relying on Release builds silently tolerating it.
  • For Error reporting: global #3, decide whether the test's expected string is stale or the print logic regressed, and update whichever is wrong.
  • Separately (tracked elsewhere): the continue-on-error: true on ctest steps in reusable-build.yml means none of this is visible from the PR checks UI — worth revisiting once these are fixed so real regressions can't hide the same way in the future.

Where this was found

Investigated while reviewing PR #2463 build logs — these failures are unrelated to that PR (same failures reproduce on master) but were flagged as worth their own issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions