Skip to content

nnUNetv2_plot_overlay_pngs: fix IndexError with more than 15 labels and wrong slice selection - #3091

Merged
FabianIsensee merged 1 commit into
MIC-DKFZ:masterfrom
Srizdebnath:fix/overlay-plot-many-labels
Nov 20, 2026
Merged

FabianIsensee merged 1 commit into
MIC-DKFZ:masterfrom
Srizdebnath:fix/overlay-plot-many-labels

Conversation

@Srizdebnath

@Srizdebnath Srizdebnath commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

nnUNetv2_plot_overlay_pngs (the tool the readers' alignment warnings point users to) has two bugs in nnunetv2/utilities/overlay_plots.py. The first makes it crash on datasets with many labels. The second makes it pick the wrong slice.

Reproduction

import numpy as np
from nnunetv2.utilities.overlay_plots import generate_overlay, select_slice_to_plot2

# a slice with background and 15 foreground labels (e.g. any TotalSegmentator-style dataset)
seg = np.repeat(np.arange(16), 4).reshape(8, 8)
image = np.random.RandomState(0).rand(8, 8)
try:
    overlay = generate_overlay(image, seg)
    print('overlay', overlay.shape, overlay.dtype)
except Exception as e:
    print(type(e).__name__, e)

# class 1 split 60/40 over slices 0 and 1, class 2 is a single voxel in slice 2
seg3d = np.zeros((3, 10, 10), dtype=np.uint8)
seg3d[0].flat[:60] = 1
seg3d[1].flat[:40] = 1
seg3d[2, 0, 0] = 2
print('selected slice:', select_slice_to_plot2(np.zeros_like(seg3d, dtype=float), seg3d))

On master:

IndexError tuple index out of range
selected slice: 0

With this PR:

overlay (8, 8, 3) uint8
selected slice: 2

I also checked it end to end with plot_overlay on a small synthetic NIfTI pair (SimpleITKIO) with 15 foreground labels in one slice. On master it fails with the same IndexError, and with the fix it writes the PNG.

Root cause and fix

  1. generate_overlay crashes when a slice has more than 15 labels. It builds mapping by enumerating the labels present in the slice and then indexes color_cycle (15 entries) with that position. Any slice that contains background plus 15 or more labels raises an IndexError, and so does plot_overlay / plot_overlay_preprocessed. Indices past the end of the cycle now wrap around the colors after the first one. The first color stays reserved for the background, so no label gets painted black. Indices that are inside the cycle map exactly as before, so overlays for datasets with up to 15 labels do not change.

  2. select_slice_to_plot2 normalizes by the wrong total. It is meant to pick the slice with the highest mean, over classes, of "fraction of this class' voxels in this slice". But it did fg_per_slice[:, i] /= fg_per_slice.sum(), which divides by the sum of the whole array. At that point that sum is the column being normalized plus all columns already normalized (each of which sums to 1). Only the first class was normalized correctly. Every later class was divided by roughly count + i, which underweights small classes most, and those are the ones this function exists to show. In the example above, slice 2 holds 100 % of class 2 (mean 0.5) and slice 0 holds 60 % of class 1 (mean 0.3). Master picks slice 0, which contains no class 2. The fix divides each column by its own sum.

Tests

nnunetv2/tests/test_overlay_plots.py:

  • an overlay of a slice with more labels than colors, and the wrap-around of colors past the cycle (background never reused)
  • the slice selection example above, and a random label map compared against the per-class fractions computed directly

Three of the four tests fail on master and all four pass with the fix. The fourth (the random label map) happens to pick the same slice on master and is there as an oracle check.

…lection

generate_overlay indexed color_cycle (15 colors) with the position of each label
among the labels present in the slice, so any slice with background plus 15 or
more labels raised an IndexError. Indices beyond the cycle now wrap around the
non-background colors.

select_slice_to_plot2 is meant to pick the slice with the highest mean fraction
of each class' voxels, but divided each class' per-slice counts by the sum over
all classes processed so far instead of by that class' own total. That
underweights every class after the first and can select a slice without the
small classes it is designed to show.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 nnU-Net Code Review

Summary

This PR fixes two bugs in nnUNetv2_plot_overlay_pngs (nnunetv2/utilities/overlay_plots.py):

  1. generate_overlay raises an IndexError when a slice contains more labels than color_cycle has entries.
  2. select_slice_to_plot2 normalises each class column by the wrong total.

It also adds unit tests for both.

Contribution Guidelines

  • Bug fix requirements: Met. The description has a minimal repro with master and PR output, explains the root cause of both bugs and describes the fix.
  • General applicability: Fine. This is a small fix to a generic diagnostic tool and adds no dataset-specific code.
  • Scope: Small and focused. No prior issue was needed.
  • Compatibility: No public signatures change. The functions are only used inside this module (the console entry point and the plot helpers). Nothing touches training, inference, plans or result files, so custom trainers are unaffected and no changelog entry is needed.

Key Findings

  • select_slice_to_plot2 fix is correct (overlay_plots.py ~L132). On master, fg_per_slice.sum() is evaluated after the earlier columns have already been normalised to 1. Column i was therefore divided by count_i + i instead of count_i. The error is small for large classes, but for small classes it is large: a 1-voxel class at i=1 scores 0.5 instead of 1.0. Dividing by fg_per_slice[:, i].sum() matches the docstring's intent. I checked the new test_small_class_is_not_underweighted by hand:

    • master picks slice 0 (0.30 vs 0.25), so the test fails there;
    • the fix picks slice 2 (0.5 vs 0.3).

    The test therefore does pin the bug.

  • Colour wrap-around is correct. 1 + (idx - 1) % (len(color_cycle) - 1) maps n → 1 and n+1 → 2, and never returns 0. The black background colour is never reused. Indices inside the cycle are unchanged, so overlays for datasets with ≤15 labels are byte-identical to master. The wrap uses the color_cycle parameter rather than the module constant, so custom cycles passed by callers are handled too.

  • Tests: They are deterministic, need no env vars or data and run fast. They match the style of the other files in nnunetv2/tests/. test_matches_per_class_fractions is a useful independent check of the intended formula.

Minor Suggestions (non-blocking)

  • Custom color_cycle of length 1: The wrap branch now raises ZeroDivisionError (% 0) where master raised IndexError. This is purely theoretical, but a short assert or a clearer error would be friendlier.
  • Pre-existing, not introduced here — label 0 absent from a slice: When mapping is None, the mapping is built from the labels present in that slice. If a slice has no background, the first foreground label gets index 0, which is black, so it is invisible. For the same reason, a given label can get different colours in different cases. That is outside this PR's scope, but it could be a follow-up. One option is to always reserve index 0 for label 0, or to build the mapping from all labels in the dataset.
  • Wrapped colours repeat: With more than 15 foreground labels, different labels now share colours. That is acceptable for a sanity-check overlay, and the docstring update documents it.

@github-actions github-actions Bot added the bug Something isn't working label Nov 20, 2026
@FabianIsensee

Copy link
Copy Markdown
Member

Thanks!
Believe it or not... 10 minutes ago I also stumbled onto fg_per_slice[:, i] /= fg_per_slice[:, i].sum() 🙈

@FabianIsensee
FabianIsensee merged commit ced6fd7 into MIC-DKFZ:master Nov 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants