torchmetrics.retrieval.RetrievalPrecisionRecallCurve plots precision calls it False Positive Rate

[carandraug]

🐛 Bug

RetrievalPrecisionRecallCurve plots precision (x axis) and recall (y axis). However, the plot calls them False and True positive rates. While True positive rate is the same as recall, false positive rate is not the same as precision.

1. why call it FPR/TPR instead of precision and recall which are the names used on the class?
2. is the FPR label on the x axis wrong or is the computation of precision a mistake and it was meant actually to compute FPR?

As a minor issue, usually, recall is on the x axis and precision on the y axis (just google precision recall curves), and would be nice if they were swapped.

To Reproduce

Precision is true_positives / (true_positives + false _positives), recall is true_positives / real_positives, and FPR is false_positives / total_population`. Given this data:

pred = torch.tensor([1.0, .9, .8, .7, .6, .5, .4, .3, .2, .1])
target = torch.tensor([1, 1, 0, 1, 1, 0, 0, 1, 0, 0], dtype=int)

We should expect:

	Precision	Recall	FPR
0	1/1	1/5	0/10
1	2/2	2/5	0/10
2	2/3	2/5	1/10
3	3/4	3/5	1/10
4	4/5	4/5	1/10
5	4/6	4/5	2/10
6	4/7	4/5	3/10
7	5/8	5/5	3/10
8	5/9	5/5	4/10
9	5/10	5/5	5/10

yet, we get

>> metric = RetrievalPrecisionRecallCurve()
>> metric.update(pred, target,indexes=torch.zeros(pred.shape, dtype=int))
>> metric.compute()
(tensor([1.0000, 1.0000, 0.6667, 0.7500, 0.8000, 0.6667, 0.5714, 0.6250, 0.5556,
         0.5000]),
 tensor([0.2000, 0.4000, 0.4000, 0.6000, 0.8000, 0.8000, 0.8000, 1.0000, 1.0000,
         1.0000]),
 tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10]))
>> fig, ax = metric.plot()
>> ax.set_xlim(0, 1)
>> ax.set_ylim(0, 1)

Environment

• TorchMetrics version 1.1.2
• Python 3.11.2
• OS: GNU/Linux Debian

[github-actions]

Hi! Thanks for your contribution! Great first issue!

[carandraug]

I can very easily submit a PR that just changes the axis label to have the x axis "Precision" . Should I also change the y axis to recall?

[carandraug]

I have submitted a PR that fixes it the label names and swap the axis. If we kept Precision on the x axis, then the line will move back and forth with multiple y values for a given x 😮 , e.g.:

Current torchmetrics version

Precision on X axis (incorrectly named FPR) and recall (also TPR) on Y axis. Line moves back and forth (note, e.g., multiple values of y for x at 0.8).

With code in PR #3404

There is the issue that swapping the axis is backward incompatible but I would argue that this is a bug to fix. There is also precedent for fixing this. The precision recall curves in classification had the same issue and the axis were swapped in torchmetrics 1.3.0 (see PR #2183).