Skip to content

Precision-Recall Curve Bugfix #862

Merged
czaloom merged 16 commits into
mainfrom
czaloom-fix-ap-12-3-25
Dec 4, 2025
Merged

Precision-Recall Curve Bugfix #862
czaloom merged 16 commits into
mainfrom
czaloom-fix-ap-12-3-25

Conversation

@czaloom

@czaloom czaloom commented Dec 3, 2025

Copy link
Copy Markdown
Collaborator

Issue

AP was inconsistent between v0.36.6 and v0.37.2 (latest) when tested on a larger real-world dataset

  • Manual inspection of the computation cache yielded the following bugs
    • calculate_ranking_boundaries would sometimes include groundtruth-prediction pairs without matching labels in the iou_boundary array. This array should only include candidates for true positives.
    • For high-density clusters of annotations, the pr_curve binning operation within compute_counts would not necessarily return the maximum precision or score depending on ordering. When running over file-based chunks this would intermittently yield different results as there was more opportunity for the correct maximum value to be chosen.

Both of these bugs resulted in small errors that could accumulate into larger errors if using file-based chunking.

Changes

  • updated calculate_ranking_boundaries to enforce label matching as a condition for inclusion
  • updated compute_counts with a new binning operation.

@czaloom czaloom self-assigned this Dec 3, 2025
@czaloom czaloom marked this pull request as ready for review December 3, 2025 22:36
@czaloom czaloom requested a review from a team as a code owner December 3, 2025 22:36
@czaloom czaloom changed the title AP Bugfix Precision-Recall Curve Bugfix Dec 4, 2025
@czaloom czaloom merged commit bdd6444 into main Dec 4, 2025
4 checks passed
@czaloom czaloom deleted the czaloom-fix-ap-12-3-25 branch December 4, 2025 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants