Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
719 changes: 719 additions & 0 deletions .bandit-baseline.json

Large diffs are not rendered by default.

62 changes: 58 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,11 @@ jobs:
- os: macos-latest
python-version: '3.12'
steps:
- uses: actions/checkout@v4
# Actions are pinned to a commit SHA, not a tag: a tag is mutable, so
# `@v4` is an unaudited third party with write access to this build.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

# tests/test_nse_integration.py shells out to real nmap against local stub
# servers. Without nmap the whole module skips itself, so installing it is
Expand All @@ -41,7 +45,7 @@ jobs:
sudo apt-get install -y --no-install-recommends nmap

- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
Expand All @@ -68,15 +72,19 @@ jobs:
matrix:
python-version: ['3.8', '3.9']
steps:
- uses: actions/checkout@v4
# Actions are pinned to a commit SHA, not a tag: a tag is mutable, so
# `@v4` is an unaudited third party with write access to this build.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Install nmap
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends nmap

- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true

Expand All @@ -85,3 +93,49 @@ jobs:
uv run --isolated --no-project --python ${{ matrix.python-version }}
--with pytest --with pytest-cov
pytest tests/ -v

# Separate job, not a step on `test`: a lint failure and a test failure are
# different questions, and neither should hide the other behind a red X.
# Ruleset and pinned ruff version live in pyproject.toml.
lint:
name: ruff
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
python-version: '3.12'
enable-cache: true

- name: Ruff
run: uv run --frozen ruff check spoonmap.py tests/

# SAST against a committed baseline: spoonmap shells out to masscan/nmap and
# parses their XML, so a bare run reports 32 reviewed findings (subprocess
# list-form calls, xml.etree on tool output we invoked ourselves) every time.
# Baselining them means only a NEW finding fails the build. Regenerate with
# uvx --from 'bandit[toml]==1.9.4' bandit -r spoonmap.py -c pyproject.toml \
# -f json -o .bandit-baseline.json
# and say in the commit message why each added finding is acceptable.
bandit:
name: bandit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
python-version: '3.12'
enable-cache: true

- name: Bandit
run: >
uvx --from "bandit[toml]==1.9.4" bandit
-r spoonmap.py -c pyproject.toml -b .bandit-baseline.json
13 changes: 13 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,19 @@ for 3.8/3.9; `requires-python` stays `>=3.8` because `spoonmap.py` itself is
dependency-free stdlib. The 95% coverage floor is enforced by pytest's `addopts`,
so every job inherits it without restating it.

Two more jobs guard style and security: `lint` runs `ruff check spoonmap.py
tests/` against a narrow `E4`/`E7`/`E9`/`F` ruleset with ruff exact-pinned in the
`dev` group (a linter that grows an opinion should not fail an unrelated PR), and
`bandit` runs SAST against the committed `.bandit-baseline.json`. `ruff format` is
deliberately **not** adopted — reformatting the module and the 11k-line test file
would bury every future diff, and `E501` alone flags 288 existing lines. The
bandit baseline holds 32 reviewed findings (list-form `subprocess` calls,
`xml.etree` parsing of masscan/nmap output we invoked ourselves), so only a *new*
finding fails; regenerate it deliberately and justify additions in the commit
message rather than adding inline `# nosec` suppressions. Actions are pinned to
commit SHAs with the tag in a trailing comment, and every checkout sets
`persist-credentials: false`.

Pass `--cleanup [dir]` to remove prior scan output non-interactively (reads `output_path` from `config.json` if no directory is given).

## Architecture
Expand Down
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -505,6 +505,31 @@ run:
with `pyproject.toml`.
- **`test-legacy`** — Python 3.8 and 3.9, resolving pytest fresh outside the
project (`uv run --isolated --no-project`).
- **`lint`** — `ruff check spoonmap.py tests/`. Run it locally with
`uv run ruff check spoonmap.py tests/`.
- **`bandit`** — SAST against the committed `.bandit-baseline.json`, so only a
*new* finding fails the build.

Actions are pinned to commit SHAs (with the tag in a trailing comment) rather
than mutable tags, and every checkout uses `persist-credentials: false`.

The ruff ruleset is deliberately narrow — `E4`, `E7`, `E9`, `F` — and the ruff
version is exact-pinned in the `dev` group so a ruff release cannot fail CI on
its own. `ruff format` is **not** used: reformatting a 4.6k-line module and an
11k-line test file would bury every future diff, and `E501` alone would flag
288 existing lines.

`bandit` reports 32 reviewed findings on `spoonmap.py` — list-form `subprocess`
calls and `xml.etree` parsing of output from tools we invoked ourselves — which
is why they are baselined instead of suppressed inline. To re-baseline after an
intentional change:

```bash
uvx --from 'bandit[toml]==1.9.4' bandit -r spoonmap.py -c pyproject.toml \
-f json -o .bandit-baseline.json
```

Say in the commit message why each newly added finding is acceptable.

The split exists because `pytest` only ships the fix for CVE-2025-71176 in
9.0.3+, which requires Python 3.10+. `uv.lock` therefore resolves for 3.10+
Expand Down
23 changes: 23 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ spoonmap = "spoonmap:main"
dev = [
"pytest>=9.0.3",
"pytest-cov>=5.0.0",
# Exact-pinned: a ruff bump can add rules, and CI failing because the linter
# grew an opinion is noise, not a signal. Bump deliberately.
"ruff==0.16.4",
]

[tool.uv]
Expand All @@ -27,6 +30,26 @@ dev = [
# and CI tests 3.8/3.9 without the lock. See .github/workflows/ci.yml.
environments = ["python_version >= '3.10'"]

[tool.ruff]
# Match the oldest interpreter the tool claims, so ruff never suggests syntax
# that would break on the jumpbox this gets run from.
target-version = "py38"

[tool.ruff.lint]
# Deliberately narrow: pyflakes (F) catches undefined names, unused imports and
# dead locals — real bugs — while E4/E7/E9 cover import placement, statement
# style and syntax errors. Formatting rules (E1/E5/W) are excluded on purpose:
# `ruff format` is not adopted here, because reformatting an 11k-line test file
# and a 4.6k-line module would bury every future diff under churn, and E501
# alone would flag 288 existing lines.
select = ["E4", "E7", "E9", "F"]

[tool.bandit]
# The scan targets spoonmap.py only. tests/ is full of deliberately hostile
# fixtures (fake XML, patched subprocess) that a SAST tool has nothing useful
# to say about.
exclude_dirs = ["tests", ".venv"]

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--cov=spoonmap --cov-report=term-missing --cov-fail-under=95"
Expand Down
10 changes: 5 additions & 5 deletions spoonmap.py
Original file line number Diff line number Diff line change
Expand Up @@ -4437,9 +4437,9 @@ def _write_findings_txt(output_path, target_scan, findings):
lines.append(f' {cmd}')
for extra in repro.get('extra_cmds', []):
base_dn = next(
(l[len('Base DN:'):].strip()
for l in grp['detail'].splitlines()
if l.startswith('Base DN:')),
(line[len('Base DN:'):].strip()
for line in grp['detail'].splitlines()
if line.startswith('Base DN:')),
'',
)
lines.append(f' {extra.format(host=hosts[0], base_dn=base_dn, port=pnum)}')
Expand Down Expand Up @@ -5572,7 +5572,7 @@ def main(): # pragma: no cover -- interactive CLI entry point; orchestrates
full_n = len(category_names) + 1
print(f'\t({full_n}) Full Port Scan [1-65535, TCP only — no UDP]')
print(f'\t(a) All Categories [{", ".join(category_names)}]')
print(f'\t(c) Custom Port Scan [enter your own comma-separated ports]')
print('\t(c) Custom Port Scan [enter your own comma-separated ports]')
print(_COLOR_INFO
+ '\nNote: Full Port Scan covers TCP 1-65535 only. It runs NO UDP '
'discovery, so the U: ports in the categories above (SNMP U:161, '
Expand Down Expand Up @@ -5894,7 +5894,7 @@ def main(): # pragma: no cover -- interactive CLI entry point; orchestrates
if effective_host_count > 0:
print(_COLOR_INFO
+ f'Work units ({work_units:,}) > threshold ({nmap_threshold:,}): '
+ f'using masscan for port discovery'
+ 'using masscan for port discovery'
+ _COLOR_RESET)
status_summary = mass_scan(
scan_type, dest_ports, source_port, max_rate,
Expand Down
31 changes: 16 additions & 15 deletions tests/test_spoonmap.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,14 @@
DISCOVERY_MASSCAN_PORTS_INTERNAL,
DISCOVERY_TCP_PORTS_INTERNAL,
EXTERNAL_PORT_SCRIPTS,
EXTERNAL_PROBE_PORT_PRIORITY,
EXTERNAL_SENSITIVE_PORTS,
HONEYPOT_MIN_PORTS_SCANNED,
HONEYPOT_MIN_UNMATCHED_PORTS,
HONEYPOT_OPEN_PORT_FRACTION,
HOST_DISCOVERY_NMAP_THRESHOLD,
INTERNAL_DISCOVERY_MAX_RATE,
INTERNAL_DISCOVERY_STATE_CEILING,
SLOW_PORTS,
SQL_SERVER_EOL,
INTERNAL_PORT_SCRIPTS,
PROBE_PORT_PRIORITY,
_build_discovery_target_file,
_build_interactive_config,
_build_nmap_cmd,
Expand Down Expand Up @@ -101,7 +97,6 @@
_scan_extra_sql_ports,
_validate_snmp_any_community,
_SMB_COUPLED_PORTS,
_SMB_PORTS,
SERVICE_CATEGORIES,
_calc_scan_wait,
_cleanup_cmd,
Expand Down Expand Up @@ -4930,7 +4925,8 @@ def test_completed_batch_skipped_when_resume_true(self, tmp_path):
targets_file.parent.mkdir(parents=True, exist_ok=True)
targets_file.write_text('10.0.0.1\n')
# Make batch XML newer than targets file
import os, time as _time
import os
import time as _time
os.utime(str(targets_file), (0, 0))
os.utime(str(batch_xml), (_time.time(), _time.time()))

Expand Down Expand Up @@ -4982,7 +4978,8 @@ def test_live_hosts_loaded_from_file_when_batch_skipped(self, tmp_path):
targets_file = tmp_path / 'discovery' / 'resolved_targets.txt'
targets_file.parent.mkdir(parents=True, exist_ok=True)
targets_file.write_text('10.0.0.1\n')
import os, time as _time
import os
import time as _time
os.utime(str(targets_file), (0, 0))
os.utime(str(batch_xml), (_time.time(), _time.time()))

Expand All @@ -5005,7 +5002,8 @@ def test_partial_resume_only_skips_completed_batches(self, tmp_path):
targets_file = tmp_path / 'discovery' / 'resolved_targets.txt'
targets_file.parent.mkdir(parents=True, exist_ok=True)
targets_file.write_text('10.0.0.1\n')
import os, time as _time
import os
import time as _time
os.utime(str(targets_file), (0, 0))
os.utime(str(batch0_xml), (_time.time(), _time.time()))

Expand Down Expand Up @@ -5037,7 +5035,8 @@ def test_batch_not_skipped_when_targets_file_is_newer(self, tmp_path):
targets_file.parent.mkdir(parents=True, exist_ok=True)
targets_file.write_text('10.0.0.1\n')

import os, time as _time
import os
import time as _time
# Make batch XML *older* than targets (simulates ranges.txt change)
os.utime(str(batch_xml), (0, 0))
os.utime(str(targets_file), (_time.time(), _time.time()))
Expand Down Expand Up @@ -6319,7 +6318,8 @@ def _run(self, tmp_path, ip_to_hostname=None, script_scan=False,
interrupt_event = threading.Event()

if popen_side_effect is None:
popen_side_effect = lambda *a, **k: self._make_finished_proc()
def popen_side_effect(*a, **k):
return self._make_finished_proc()

with patch('spoonmap._build_nmap_cmd', return_value=['nmap', 'fake']) as mock_build, \
patch('spoonmap._get_scripts_for_port', return_value=scripts_for_port), \
Expand Down Expand Up @@ -8673,12 +8673,13 @@ def test_hosts_on_139_propagate_to_445(self, tmp_path):
}
with patch('spoonmap._run_masscan_batch',
side_effect=self._make_batch_side_effect(responses)):
result = mass_scan('All', ['139', '445'], '88', '1000',
'/fake/targets.txt', '', batch_size=5)
mass_scan('All', ['139', '445'], '88', '1000',
'/fake/targets.txt', '', batch_size=5)

port445_file = tmp_path / 'discovery' / 'live_hosts' / 'port445.txt'
if port445_file.exists():
written = {l.strip() for l in port445_file.read_text().splitlines() if l.strip()}
written = {line.strip() for line in port445_file.read_text().splitlines()
if line.strip()}
assert written == {'10.0.0.1', '10.0.0.2', '10.0.0.3'}

def test_smb_coupled_ports_constant(self):
Expand Down Expand Up @@ -9477,7 +9478,7 @@ def test_udp_ports_not_passed_to_masscan(self, tmp_path):
"""dest_ports with U:500 → masscan never called with U:500."""
spoonmap.output_path = str(tmp_path)
with patch('spoonmap._run_masscan_batch', return_value={}) as mock_m, \
patch('spoonmap._nmap_udp_discovery', return_value=set()) as mock_u:
patch('spoonmap._nmap_udp_discovery', return_value=set()):
mass_scan('All', ['443', 'U:500'], '53', '10000',
'/fake/targets.txt', '', batch_size=1)
for call in mock_m.call_args_list:
Expand All @@ -9487,7 +9488,7 @@ def test_udp_ports_not_passed_to_masscan(self, tmp_path):
def test_udp_ports_trigger_nmap_udp_discovery(self, tmp_path):
"""dest_ports with U:500 → _nmap_udp_discovery called with 'U:500'."""
spoonmap.output_path = str(tmp_path)
with patch('spoonmap._run_masscan_batch', return_value={}) as mock_m, \
with patch('spoonmap._run_masscan_batch', return_value={}), \
patch('spoonmap._nmap_udp_discovery', return_value=set()) as mock_u:
mass_scan('All', ['443', 'U:500'], '53', '10000',
'/fake/targets.txt', '', batch_size=1)
Expand Down
Loading
Loading