Fix FROC num_targets miscount when labels_to_exclude is set#8951
Fix FROC num_targets miscount when labels_to_exclude is set#8951aymuos15 wants to merge 4 commits into
Conversation
compute_fp_tp_probs_nd derived the target count as max_label minus the length of labels_to_exclude, which is wrong whenever the exclusion list contains a label that is absent from the mask, is greater than max_label, or is duplicated. In those cases num_targets is too small and inflates sensitivity downstream in compute_froc_curve_data, where total_tps is divided by num_targets. Count instead the labels in [1, max_label] that are not excluded, reusing the existing loop so an excluded or out-of-range entry can no longer subtract a target that was never counted. Signed-off-by: Soumya Snigdha Kundu <soumya_snigdha.kundu@kcl.ac.uk>
Cover an out-of-range exclusion and a duplicated exclusion; both previously reduced num_targets below the true target count and fail before the fix. Signed-off-by: Soumya Snigdha Kundu <soumya_snigdha.kundu@kcl.ac.uk>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthrough
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
monai/metrics/froc.py (1)
70-75: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winCanonicalize
labels_to_excludebefore the loop.This fix is correct, but
if i not in labels_to_excludenow does a linear scan for every label. Converting once to a set keeps the duplicate-handling behavior explicit and avoids the extra O(max_label * len(labels_to_exclude)) cost.Proposed change
if labels_to_exclude is None: labels_to_exclude = [] + excluded_labels = set(labels_to_exclude) max_label = np.max(evaluation_mask) tp_probs = np.zeros((max_label,), dtype=np.float32) @@ fp_probs = probs[np.where(hittedlabel == 0)] num_targets = 0 for i in range(1, max_label + 1): - if i not in labels_to_exclude: + if i not in excluded_labels: num_targets += 1 if i in hittedlabel: tp_probs[i - 1] = probs[np.where(hittedlabel == i)].max()As per path instructions, "Suggest any enhancements for code improving efficiency, maintainability, comprehensibility, and correctness."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@monai/metrics/froc.py` around lines 70 - 75, The loop in froc’s target counting still checks labels_to_exclude with repeated linear membership tests, so canonicalize labels_to_exclude once before the loop and use that normalized form inside the max_label iteration. Update the logic around num_targets and the hittedlabel/probs assignment to reference the canonicalized exclusion collection, keeping the duplicate-handling behavior explicit while improving efficiency and readability.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@monai/metrics/froc.py`:
- Around line 70-75: The loop in froc’s target counting still checks
labels_to_exclude with repeated linear membership tests, so canonicalize
labels_to_exclude once before the loop and use that normalized form inside the
max_label iteration. Update the logic around num_targets and the
hittedlabel/probs assignment to reference the canonicalized exclusion
collection, keeping the duplicate-handling behavior explicit while improving
efficiency and readability.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 14342841-a2a9-46fb-a813-505e90ed40ac
📒 Files selected for processing (2)
monai/metrics/froc.pytests/metrics/test_compute_froc.py
num_targets is now a plain Python int counter, so the cast(int, ...) wrapper is flagged by mypy as a redundant-cast. Signed-off-by: Soumya Snigdha Kundu <soumya_snigdha.kundu@kcl.ac.uk>
Description
compute_fp_tp_probs_ndcomputednum_targets = max_label - len(labels_to_exclude), which only holds when every excluded label is a distinct value present in[1, max_label]. An absent, out-of-range, or duplicated entry subtracts targets that were never counted, leavingnum_targetstoo small. Sincecompute_froc_curve_datadivides cumulative true positives bynum_targets, this inflates the reported sensitivity.The count is now the labels in
[1, max_label]that are not excluded, computed in the existing loop. A regression test covering an out-of-range and a duplicated exclusion is included; both undercount before the fix.Types of changes
./runtests.sh -f -u --net --coverage../runtests.sh --quick --unittests --disttests.