Skip to content

stat_scores.py - top_k not validated for multiclass classification metrics #3405

Description

@leopozh

🐛 Bug

top_k is not validated for the multiclass classification metrics, negative integers and floating points are accepted instead of raising a ValueError

The check on line 244 of src/torchmetrics/functional/classification/stat_scores.py uses and

if not isinstance(top_k, int) and top_k < 1:
    raise ValueError(f"Expected argument `top_k` to be an integer larger than or equal to 1, but got {top_k}")

So not isinstance(top_k, int) and top_k < 1 is only True when top_k is both not an int and less than 1. Checked this and isinstance(-1, int) becomes true. A floating point variable like 4.5 passes because 4.5 < 1 is False

ValueError never gets raised here

To Reproduce

Calling any multiclass metric with top_k = -1 or top_k = 4.5 does not raise ValueError.

import torch
from torchmetrics.classification import MulticlassAccuracy

preds = torch.randn(10, 5)
target = torch.randint(5, (10,))

MulticlassAccuracy(num_classes=5, top_k=-1) # no error gets raised for this case

MulticlassAccuracy(num_classes=5, top_k=4.5) # no error gets raised for this case

Expected a ValueError is raised for any top_k that is not a positive int

TorchMetrics version: 1.9.0, built from source, commit 47223c1
Python & PyTorch Version: Python 3.9.6, PyTorch 2.8.0
OS: macOS, Apple Silicon (M1 Max)

Additional context:
Same validation issue as #[3378] (retrieval metrics), being fixed in #3379

That PR only changes the retrieval files and does not touch src/torchmetrics/functional/classification/stat_scores.py.

Happy to submit a pr :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug / fixSomething isn't workinghelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions