A practitioner's field guide to Infrared Small Target Detection (IRSTD) / Infrared Small Target Segmentation (ISTD): the datasets, methods, and benchmarks that matter, organized so you can actually decide what to use instead of just scrolling a link dump.
Compiled from hands-on ML research in this domain. Started with datasets because "which dataset should I train/evaluate on" is the question almost everyone asks first - and the one existing lists answer worst.
Keywords: infrared small target detection, infrared small target segmentation, ISTD, IRSTD, dim-small target detection.
- Why this exists
- Datasets
- Methods - coming soon
- Benchmarks & Metrics
- Recommended Surveys
- Contributing / Contact
Most existing "awesome IRSTD" lists are excellent at completeness (every paper, every year) but weak at orientation - a newcomer or even an experienced researcher switching sub-problems still can't tell, at a glance:
- Is this dataset real or synthetic?
- Single target per image, or dense/clustered?
- What annotation do I actually get - a mask, a box, or just a point?
- Is it big enough to pretrain on, or only useful as a held-out test set?
- Ground/sky/sea/space - does the background match my use case?
This repo answers those questions first, with links second.
| Your situation | Use | Why |
|---|---|---|
| "I need the standard benchmark everyone compares against" | SIRST, NUDT-SIRST, IRSTD-1k | The three most-cited single-frame benchmarks; almost every SOTA paper reports on this trio, so you get free comparability |
| "I want to pretrain / need lots of clean data" | NUDT-SIRST, MDFA (train split) | Synthetic target injection into real backgrounds → unlimited variety, pixel-perfect masks, no annotation noise |
| "I need real-world, messy, field-like clutter" | IRSTD-1k | Fully real infrared images with manual pixel-level annotation; harder and more representative of deployed conditions than synthetic sets |
| "I'm benchmarking weak supervision (box/point-only)" | SIRST v2 | Ships with mask and bounding-box and point annotations for the same images |
| "I care about dense/clustered targets or sky context" | DenseSIRST | Adds sky-segmentation labels so you can study target-background relationships, not just target-only masks |
| "I need to test cross-domain / cross-sensor generalization" | WideIRSTD-Full | Aggregates multiple domains/sensors with auxiliary metadata instead of one homogeneous source |
| "My targets are ships on sea backgrounds (space-based sensor)" | NUDT-SIRST-Sea | Purpose-built sea-only background, ship-shaped targets |
| "I'm working on moving-target / multi-frame / video models" | IRDST, NUDT-MIRSDT, DMIST | Purpose-built temporal sequences with motion cues, not single frames |
| "I'm doing satellite video / long-range jitter scenarios" | IRSatVideo-LEO | Modeled specifically on LEO satellite video artifacts (jitter, long range, cloud interference) |
| "I need to stress-test robustness to sensor/background noise" | NUDT-MIRSDT-Noise | Deliberately noised variant of NUDT-MIRSDT for robustness evaluation |
| Type | What you get | Good for |
|---|---|---|
| Pixel mask | Per-pixel binary segmentation map | Segmentation-quality metrics (IoU, nIoU, Pd/Fa at pixel level) |
| Bounding box | Rectangle around each target | Detection-style pipelines, weak-supervision research |
| Point / centroid | Single (x, y) per target | Single-point-supervision research (e.g. label-evolution / pseudo-mask methods) |
Several newer datasets (SIRST v2, WideIRSTD-Full) ship more than one annotation type for the same images - check before assuming a dataset is mask-only.
| Dataset | Year | Domain | Source | Scale | Resolution | Annotation | Links |
|---|---|---|---|---|---|---|---|
| MDFA | 2019 | Sky / Ground / Sea | Synthetic (composite) | 10,000 train + 100 test images | Varies | Pixel mask |   |
| SIRST | 2021 | Sky / Ground / Sea | Real | 427 images, 480 target instances | Varies | Pixel mask | |
| SIRST-Aug | 2021/2023 | Sky / Ground / Sea | Real (augmented split of SIRST) | Larger than SIRST via augmentation | Varies | Pixel mask |  |
| NUDT-SIRST | 2022 | Sky / Ground / Sea / Highlight / Cloud (5 scene categories) | Synthetic (target injection into real backgrounds) | 1,327 images | 256×256 | Pixel mask | |
| IRSTD-1k | 2022 | Mixed real-world (drones, vehicles, ships, etc.) | Real | 1,000 images | 512×512 | Pixel mask (manual) | |
| SIRST v2 | 2023 | Sky / Ground / Sea | Real (extension of SIRST) | 512 images | Varies | Mask + bounding box + point | |
| NUDT-SIRST-Sea | 2023 | Sea only (space-based sensor) | Synthetic/real ship imagery | Focused ship-detection subset | Varies | Pixel mask / box | |
| DenseSIRST | 2024 | Sky-heavy, clustered targets | Real | Extends SIRST-family with sky-semantic labels | Varies | Pixel mask + sky segmentation | |
| WideIRSTD-Full | 2025 | Multi-domain / multi-sensor (omni-domain) | Mixed real, aggregated | Large-scale (multiple sub-domains) | Varies | Mask + metadata | |
| Dataset | Year | Domain | Scale | Annotation | Notes | Links |
|---|---|---|---|---|---|---|
| Hui et al. | 2020 | Air / ground background, dim-small aircraft | Multiple sequences | Detection + tracking labels | Early public sequence dataset for aircraft-type dim targets |  |
| Fu et al. | 2021 | Air-ground, time-sensitive targets | Multiple sequences | Detection + tracking labels | Focused on time-sensitive target scenarios | |
| SAITD | 2021 | Cluttered background | Multiple sequences | Moving-target labels | Small infrared moving targets under heavy clutter | |
| IRDST | 2023 | Sky / ground / air, real + simulated | Large-scale | Bounding box | Built alongside RDIAN; one of the larger public multi-frame sets | |
| NUDT-MIRSDT | 2023 | Mixed synthetic + real, moving targets | Multiple sequences | Pixel mask | Companion to DTUM; supports per-frame segmentation of moving targets | |
| SIRSTD | 2024 | Sequential imagery | Multiple sequences | Pixel mask | Built for spatial-temporal transformer models | |
| TSIRMT | 2024 | Sequential moving targets | Multiple sequences | Pixel mask / box | Used to benchmark LMAFormer-style local-motion models | |
| DMIST | 2024 | Dense moving targets, mixed backgrounds | Large-scale | Pixel mask | Explicitly targets dense moving small-target scenes, unlike sparser earlier sets | |
| IRSatVideo-LEO | 2025 | Satellite video (LEO orbit) | Multiple long sequences | Pixel mask | Models satellite-specific artifacts: platform jitter, long imaging range, cloud interference | |
| NUDT-MIRSDT-Noise | 2026 | Same as NUDT-MIRSDT, with injected noise | Multiple sequences | Pixel mask | Robustness-testing variant - use alongside NUDT-MIRSDT, not instead of it | |
- Synthetic ≠ free lunch. MDFA and NUDT-SIRST give you clean masks and scale, but models tuned only on injected-target synthetics can overfit to unrealistically sharp target/background contrast. Validate on a real set (IRSTD-1k, SIRST) before trusting synthetic-only numbers.
- "Single-frame" benchmarks are not interchangeable. SIRST, NUDT-SIRST, and IRSTD-1k differ in resolution, target density, and background diversity - a method ranking #1 on one does not guarantee the same on another. Report all three if you can.
- Multi-frame ≠ just single-frame stacked. Motion-based methods (DTUM, temporal-difference approaches) need datasets with genuine target displacement across frames - check the dataset paper explicitly states inter-frame motion, not just repeated static crops.
- Metadata matters for domain-generalization claims. If you're claiming cross-domain robustness, use WideIRSTD-Full or explicitly train/test across two different single-domain sets - a single-domain dataset can't support that claim alone.
Coming soon - will cover background-suppression, HVS-based, optimization/tensor-based, deep-learning, and deep-unfolding methods, each with the same "when should I actually use this" framing as the datasets above.
IRSTD is one of the few sub-fields where you genuinely cannot reuse mainstream CV evaluation off the shelf. Understanding why the field invented its own metrics is as important as knowing the formulas - it's what lets you tell a rigorous benchmark table from a misleading one.
- COCO-style mAP@IoU assumes objects have area. IRSTD targets are frequently 1-9 pixels, sometimes sub-pixel after blur. A prediction that's off by a single pixel can already fail an IoU=0.5 box/mask match - so box-IoU matching either rejects almost everything or has to be loosened to the point of being meaningless. This is why the field matches targets by centroid distance instead of IoU overlap (see Pd/Fa below).
- Dataset-pooled mIoU is dominated by background. A typical IRSTD image is >99.9% background pixels. If you pool TP/FP/FN counts across the whole dataset before dividing (standard semantic-segmentation mIoU), a method that misses small targets entirely but gets the (huge) background right still scores deceptively high. IRSTD papers instead compute IoU only over the foreground/target regions, and separately track false alarms.
- A handful of large targets can dominate an average. Even with foreground-only IoU, if you average pixel counts instead of per-sample scores, images with bigger targets (more pixels) swamp images with genuinely dim, 2-3 pixel targets. This is precisely why nIoU exists (below) - it was proposed for this field specifically, not borrowed from elsewhere.
| Metric | Level | Formula | What it tells you |
|---|---|---|---|
| IoU | Pixel, dataset-pooled | IoU = ΣTPᵢ / (ΣTPᵢ + ΣFPᵢ + ΣFNᵢ), summed over all i samples before dividing | Overall pixel-level overlap quality - but can be skewed by a few large targets |
| nIoU (normalized IoU) | Pixel, per-sample averaged | nIoU = (1/N) Σᵢ TPᵢ/(TPᵢ+FPᵢ+FNᵢ) - average the ratio, not the counts | The field's fix for IoU's large-target bias; standard alongside IoU since the ACM paper (Dai et al., WACV 2021) |
| Precision / Recall / F1 | Pixel or target | Standard definitions, applied per-pixel or per-matched-target | Useful for weak-supervision (box/point) work where mask overlap isn't the point |
| Pd (Probability of Detection) | Target | Pd = (# correctly matched targets) / (# ground-truth targets) | "Did we find it at all" - matching is via centroid distance, not overlap |
| Fa (False-Alarm Rate) | Target | Fa = (# false-positive detections) / (# image pixels) - reported in scientific notation, e.g. 10⁻⁵ | How much the operator gets spammed with false tracks - see why this matters operationally below |
| ROC (Pd vs. Fa) + AUC | Target | Sweep the decision threshold, plot Pd against Fa | The honest way to characterize a detector - a single (Pd, Fa) point at one threshold is cherry-pickable |
| 3D-ROC family | Target | Adds threshold τ as a 3rd axis; derived surfaces/AUCs: (τ,TPR), (τ,FPR), TD, BS, TDBS, SNPR, ODP | Separates how well a method suppresses background (BS) from how well it preserves the target (TD) instead of collapsing both into one ROC point - used in classical/tensor-method papers, implemented in TensorISTD |
| SCRG (SCR Gain) | Patch / classical methods | SCRG = SCR_out / SCR_in, where SCR = |μ_target − μ_background| / σ_background | How much a filter/enhancement step improved local target-vs-clutter contrast - legacy metric from HVS/filter-based methods, rarely reported for deep nets |
| BSF (Background Suppression Factor) | Patch / classical methods | BSF = σ_background,in / σ_background,out | How much local background clutter variance was suppressed - companion to SCRG |
| Params / FLOPs / FPS | Efficiency | Model size, compute cost, measured latency on stated hardware | The metric academic leaderboards most often omit and deployment engineers care about most |
| Toolkit | Year | Language | Scope | What you actually get | Link |
|---|---|---|---|---|---|
| ISTD-python | 2021 | Python | Classical detectors only (Tophat, Fast Saliency, MaxMedian, LCM, HB-MLCM, MPCM, IPI, PSTNN) | AUC/Pd/Fa benchmarked on NUDT-SIRST, IRSTD-1K, SIRST-Aug - a classical-method baseline without touching MATLAB | |
| BasicIRSTD | 2023 | PyTorch | Unified train/test harness for classical (Top-Hat, MaxMedian, RLCM, WSLCM, TLLCM, MSLCM, MSPCM, IPI, NRAM, RIPT, PSTNN, MSLSTIPT) and deep methods (ACM, ALCNet, ISNet, RDIAN, DNANet, ISTDU-Net, UIU-Net) | IoU/Pd/Fa @ threshold 0.5 on SIRST, NUDT-SIRST, IRSTD-1K, plus cal_params.py for parameter counts - the closest thing to a one-stop reproducibility harness; most new single-frame papers already build on it |
|
| ISTD-Benchmark | 2023 | MATLAB + Python | Benchmark platform over IRSTD-1K / MDFA / SIRST-Aug | PR curves, F1, ROC with plotting, plus a detector/ folder to plug in your own model |
 |
| STD-EvalKit | 2024 | Python | Metrics-only library (no detectors) | Pixel-level (ROC/AUC, AP, PR, F1, IoU), target/center-level (Pd_Fa, Center-nIoU, AP), and box-level COCO-style mAP - unified update()/get() API, multithreaded |
|
| TensorISTD | 2025 | MATLAB | Tensor/matrix optimization detectors (PSTNN single-frame; 4D-TT/TR, LogTFNN, NPSTT, STPA-FCTN multi-frame) | SCRG, BSF, CG, 3D-ROC, and 8 AUC variants (FPR/TPR, τ/TPR, τ/FPR, ODP, SNPR, TD, BS, TDBS) + runtime - the deepest classical/tensor-method eval suite, if you're benchmarking against optimization-based methods | |
| PyIRSTDMetrics | 2025 | Python (pip install pyirstdmetrics) |
Metrics-only library | Pixel-level IoU variants, F1/Precision/Recall/TPR/FPR, target-level Pd/Fa, hierarchical IoU - verified against BasicIRSTD's numbers, so it's a safe default metrics backend to drop into your own training loop | |
Picking one:
- Reproducing a specific paper's reported numbers → BasicIRSTD (most new single-frame work is already built on it, so thresholds/splits match).
- Just need metrics inside your own training loop → PyIRSTDMetrics or STD-EvalKit (no detector code to fight with).
- Benchmarking classical/filter methods without MATLAB → ISTD-python; if you need the full SCRG/BSF/3D-ROC suite → TensorISTD (MATLAB).
- Need box-level, COCO-style comparison for a detection-style (not segmentation-style) model → STD-EvalKit.
A benchmark table you can actually trust should report:
- Both IoU and nIoU - reporting only one hides which effect (background dilution vs. large-target bias) is driving the number.
- A full ROC curve or AUC, not a single (Pd, Fa) point at an unstated threshold - a single point is trivially cherry-picked.
- The official train/test split used (SIRST, NUDT-SIRST, and IRSTD-1K all have community-standard splits shipped with their repos/BasicIRSTD) - splits vary enough across papers that "SOTA on IRSTD-1K" numbers are not always comparable.
- Params, FLOPs, and measured FPS on stated hardware - a 2-point nIoU gain from a 5× larger model is not usually a win.
- Mean ± std over multiple seeds, especially on small sets (SIRST is 427 images) - single-run deltas of a point or two are frequently within run-to-run noise.
Academic leaderboards optimize for average-case Pd/nIoU. Operational systems live and die by a different slice of the same curves:
- Fa dominates usability, not Pd. A detector with 99% Pd but a high false-alarm rate floods an operator (or an autonomous downstream tracker) with spurious detections every few frames - in practice it gets tuned down or ignored. Look at Pd at a fixed, low Fa (e.g. Fa ≤ 10⁻⁵), not peak Pd, when judging fitness for a surveillance/seeker loop.
- Params/FLOPs/FPS gate deployability, independent of accuracy. IRSTD models frequently need to run on gimbal-mounted or embedded EO/IR payload compute, not a workstation GPU - the best nIoU on a benchmark leaderboard is irrelevant if it can't hit frame rate on the target hardware.
- Single-dataset SOTA overstates robustness. Field sensors, altitudes, and backgrounds rarely match a training distribution exactly. Cross-domain evaluation (train on one dataset, test on another; or use WideIRSTD-Full's multi-domain design) is a much better proxy for real deployment than another point of nIoU on the same dataset a method was tuned on.
- Evaluation methodology itself is an open problem. Enhancing Evaluation Methods for Infrared Small-Target Detection in Real-world Scenarios (arXiv 2301.03796) shows that standard Pd/Fa can visibly contradict qualitative visual assessment (e.g. ranking a clearly worse classical filter above a clearly better one), because threshold sensitivity and denominator choice for Fa aren't standardized across papers. Don't treat a reported Pd/Fa number as ground truth without checking how it was computed.
- Single-Frame Infrared Small-Target Detection: A survey. M. Zhao, et al. GRSM, 2022.
- Infrared small target segmentation networks: A survey. R. Kou, et al. PR, 2023.
- Deep learning based infrared small object segmentation: Challenges and future directions. Z. Yang, et al. Information Fusion, 2025.
This is a living document, built from personal research notes. If you spot an error, a missing dataset, or a link that's gone stale, open an issue or a PR.
Note:
marks the paper.
marks a dataset-only download (no associated code).
/
/
mark official code/data repos, labeled by language.