Skip to content

Add greppable output (spoonmap_output.gnmap) - #38

Merged
bandrel merged 1 commit into
mainfrom
feat/gnmap-output
Aug 21, 2026
Merged

bandrel merged 1 commit into
mainfrom
feat/gnmap-output

Conversation

@bandrel

@bandrel bandrel commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Closes #36. Implements the idea from PR #11 (Feb 2021, @zenfosec), which I closed as unmergeable.

What this adds

Each port's banner pass writes nmap_results/portN.gnmap via -oG, and those are merged into spoonmap_output.gnmap once the workers join:

# SpooNMAP aggregated greppable output
Host: 10.0.0.5 (web01)	Ports: 22/open/tcp//ssh//OpenSSH 8.9p1/, 80/open/tcp//http//nginx 1.18/
Host: 10.0.0.200 ()	Ports: 445/open/tcp//microsoft-ds///
# SpooNMAP done: 2 hosts

Why not #11's approach

#11 pointed every port at a single shared -oG file with --append-output. That was safe when nmap ran serially. nmap_scan() now runs five concurrent workers by default, so five nmap processes would be appending to one file simultaneously and interleaving their writes — producing a plausible-looking but corrupt host list. For a format whose entire purpose is machine consumption, silently corrupt is worse than absent. One file per port gives each a single writer.

Merged, not concatenated

A host open on three ports is scanned by three separate nmap runs, so it appears in three per-port files. Grepable output is one line per host, so those are merged rather than concatenated. Concatenating would make anything counting lines over-count hosts, and would put this artifact at odds with spoonmap_output.xml/.json, which merge the same hosts.

Hosts come out in numeric IP order (_ip_sort_key()) and port entries in numeric port order, so the file is stable across runs and diffable between scans.

Two consistency details worth review

A quarantined port's .gnmap is quarantined too. _quarantine_failed_output() renames the sibling .gnmap alongside the .xml. Without that, a port excluded from spoonmap_output.xml for a failed nmap exit would still contribute its partial hosts to spoonmap_output.gnmap — the two artifacts would disagree about what was scanned, and the greppable one would be the optimistic of the two. There's a test asserting the quarantined data doesn't reach the aggregate.

Ports with .xml but no .gnmap are named, not dropped. That's what resuming a scan from before this feature looks like. An artifact that quietly covers less than the run did is a coverage claim the scan never made, so it warns and points at spoonmap_output.xml. port{N}_sql.xml is excluded from that check — _scan_extra_sql_ports() writes it with no -oG, so it isn't a gap.

Safety for existing readers

Adding .gnmap files to nmap_results/ doesn't disturb anything: all three call sites that scan that directory filter on an .xml extension, and _parse_result_xml() returns None for anything else (no warning spam). spoonmap_output.gnmap is registered in _RESULT_FILES, so --cleanup removes it.

Scope is the banner pass only. The NSE script pass writes to its own directory and its results are surfaced as findings, so a -oG there would duplicate without adding anything greppable.

Verification

  • 939 passed / 5 skipped / 100% coverage (gate is 95%), 32 new tests.
  • No suppressions added.
  • The parser was checked against real nmap 7.99 -oG output, not just fixtures — confirming the tab-separated field layout, the Status: Up line nmap emits alongside the ports line for the same host, the trailing Ignored State field, and the comment header/footer. The fixtures were then written to match what nmap actually produces.

🤖 Generated with Claude Code

@bandrel

bandrel commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Note on merge order: this and #37 are independent features branched from the same main, and both show as mergeable individually, but they do conflict with each other in tests/test_spoonmap.py — both append a test block at the end of the file and both add entries to the from spoonmap import (...) list. I verified this with git merge-tree; spoonmap.py and README.md merge cleanly between the two.

So whichever of #37/#38 merges second needs a trivial conflict resolution — keep both test blocks, keep both import lines. Happy to rebase whichever one you'd rather take second; just say which and I'll push the update.

Closes #36. Originally proposed in PR #11 (Feb 2021), closed as unmergeable.

Each port's banner pass now writes nmap_results/portN.gnmap via -oG, and those
are merged into spoonmap_output.gnmap after the workers join.

Why not #11's approach: it pointed every port at one shared -oG file with
--append-output. That was safe when nmap ran serially, but nmap_scan() now runs
five concurrent workers, so five processes would interleave their appends into a
plausible-looking but corrupt host list — the worst outcome for a format whose
whole purpose is machine consumption. One file per port gives each a single
writer.

Merged, not concatenated: a host open on three ports is scanned by three nmap
runs and so appears in three files, but grepable output is one line per host.
Concatenating would make anything counting lines over-count hosts and would
disagree with spoonmap_output.xml/.json, which merge the same hosts. Hosts are
ordered by _ip_sort_key() and port entries numerically, so the file is stable
across runs and diffable between scans.

Two consistency details:

- _quarantine_failed_output() now renames the sibling .gnmap too. Otherwise a
  port excluded from spoonmap_output.xml would still contribute its partial
  hosts to spoonmap_output.gnmap, leaving the greppable artifact the more
  optimistic of the two about what was actually scanned.
- Ports holding an .xml with no .gnmap — what a resume from before this feature
  looks like — are named in a warning instead of being silently absent. An
  artifact that quietly covers less than the run did is a coverage claim the
  scan never made. port{N}_sql.xml is excluded from that check, since
  _scan_extra_sql_ports() writes it with no -oG and it is not a gap.

Adding .gnmap to nmap_results/ is safe for existing readers: all three that scan
the directory filter on an .xml extension, and _parse_result_xml() returns None
for anything else. spoonmap_output.gnmap is registered in _RESULT_FILES so
--cleanup removes it.

The parser was verified against real nmap 7.99 -oG output, not just a fixture:
tab-separated fields, the 'Status: Up' line emitted alongside the ports line,
the trailing 'Ignored State' field, and the comment header/footer.

939 tests pass (32 new), 100% coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bandrel
bandrel force-pushed the feat/gnmap-output branch from 597f66c to 589ac1d Compare August 21, 2026 16:29
@bandrel
bandrel merged commit 3e719aa into main Aug 21, 2026
7 checks passed
@bandrel
bandrel deleted the feat/gnmap-output branch August 21, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No greppable (-oG) output: add spoonmap_output.gnmap

1 participant