🕵️ Passive Recon — Enterprise OSINT/EASM/CTEM Agent

Zero-touch, purely passive external asset discovery platform

No probes sent to targets. No logs generated on their side. Just 15+ public data sources cross-validated.

🇨🇳 中文版

Status: Actively maintained — daily data source updates and CI runs on every commit.

---

🌟 What is Passive Recon?

Passive Recon is a purely passive OSINT/EASM/CTEM platform that discovers an organization's external assets by querying 15+ public data sources — certificate transparency logs, DNS records, search engines, network mapping, GitHub, Wayback Machine, and more — without ever touching the target's systems.

One command, any target:

pip install -r requirements.txt
python cli.py collect "Tsinghua University"
python cli.py collect --domain example.com "Acme Corp"

The system auto-infers the target domain, runs all 15 sources in parallel, and produces a comprehensive asset report with risk findings.

---

✨ Features

Capability	Description
Passive Asset Discovery	15+ cross-validated data sources, auto domain inference
Subdomain Enumeration	crt.sh, HackerTarget, URLScan, Wayback, DNSDumpster, and more
IP & Port Mapping	C-segment clustering, exposed port detection
Risk Detection	VPN exposure, OA systems, weak ciphers, known vulnerabilities
Compliance Guardrail	R1 compliance check on every outbound call — fail-closed
Rate Limiting	Per-IP sliding window ≤95%, queue never drops tasks
Approval Workflow	High-risk outbound requires manual approval
Audit Trail	Full operation audit log
Static Guard	CI scan that blocks active-scan code from entering production
Web Dashboard	python cli.py serve — one-click panel
Scheduled Tasks	python cli.py schedule --targets targets.txt — daily auto-collection
Auto-save Reports	Markdown report saved to data/report_<target>_<domain>.md
🤖 AI Domain Inference	DeepSeek-powered domain inference for any target (no lookup table needed)
🤖 AI Risk Scoring	Automatic risk scoring (0-100) with false positive filtering
🤖 AI Report Summary	Auto-generated analysis report after each collection
🤖 AI Chat Query	python cli.py ask "What VPNs does Tsinghua have?" — natural language asset search
📌 CVE Vulnerability Intel	Auto-correlate discovered assets with NVD/OSV for known vulnerabilities
📊 CVE Lookup	python cli.py cve CVE-2024-xxxx — query CVE details from NVD + OSV
🔁 Change Tracking	Scheduled runs compare snapshots and flag new/changed assets

---

🤖 AI Features

Passive Recon integrates with DeepSeek (no additional cost, free tier available) to provide AI-powered enhancements:

Feature	Command	Description
AI Domain Inference	python cli.py collect "any target"	Auto-infers domain for any target via AI, not just lookup table entries
AI Risk Scoring	python cli.py collect "target"	Scores each risk 0-100, filters false positives, suggests fixes
AI Report Summary	python cli.py collect "target"	Generates a natural language analysis report after collection
AI Chat Query	python cli.py ask "question"	Ask natural language questions about your asset database

Examples:

# AI domain inference works for any target, not just known ones
python cli.py collect "some unknown company"

# AI risk scoring with severity visualization
python cli.py collect "Tsinghua University"
# Output: ████████ 85 [P1] VPN entry exposed: vpn.tsinghua.edu.cn
#               💡 Suggest restricting VPN entry to an IP allowlist and enabling MFA

# AI chat query — ask questions in natural language
python cli.py ask "What VPNs does Tsinghua University have?"
python cli.py ask "List all discovered mail servers"
python cli.py ask "What are the most critical risks?"

To skip AI processing (faster, no API call):

python cli.py collect "target" --no-ai

Environment variable: Set DEEPSEEK_API_KEY (already configured if you have it) to enable AI features. The free DeepSeek tier is sufficient for personal use.

---

📡 Data Sources

Source	Key Required	Type
crt.sh	❌ Free	Certificate Transparency
HackerTarget	❌ Free	DNS / Subdomain
URLScan.io	❌ Free	Historical Snapshots
AlienVault OTX	❌ Free	Passive DNS / Threat Intel
Wayback Machine	❌ Free	Historical URLs / Subdomains
DNSDumpster	❌ Free	DNS Mapping / MX/NS Records
CommonCrawl	❌ Free	Web Crawl Archive
GitHub	❌ Free (rate-limited)	Code Leak Search
Hunter (Yingtu)	✅ Required	Network Space Mapping
FOFA	✅ Required	Network Space Search
SecurityTrails	✅ Required	Subdomain / Passive DNS
Shodan	✅ Required	Internet Device Search
VirusTotal	✅ Required	Passive DNS / Subdomain
ZoomEye	✅ Required	Network Space Mapping
Qichacha	✅ Required	Chinese Enterprise Registry
NVD	❌ Free	CVE Vulnerability Database
OSV.dev	❌ Free	Open Source Vulnerability Database

---

🚀 Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Configure API Keys (choose one)

Option A: Environment Variables (recommended — never written to disk)

# Linux / macOS
export PASSIVE_API_KEYS='{"hunter":["key1","key2"],"qichacha":{"app_key":"xxx","secret_key":"xxx"}}'

# Windows PowerShell
$json='{"hunter":["key1","key2"],"qichacha":{"app_key":"xxx","secret_key":"xxx"}}'
[Environment]::SetEnvironmentVariable('PASSIVE_API_KEYS', $json, 'User')

Option B: config.json

cp config.example.json config.json
# Edit config.json with your keys

Zero-config mode: run python cli.py serve --demo to start with mock data — no API keys needed for evaluation.

3. Run Asset Discovery

# One-shot: any target, auto domain inference
python cli.py collect "Tsinghua University"
python cli.py collect --domain example.com "Acme Corp"

# Batch mode
python cli.py batch targets.txt

# One-click web dashboard
python cli.py serve

# Daily scheduled collection (2:00 AM)
python cli.py schedule --targets targets.txt

---

📊 Sample Output

After running python cli.py collect "Tsinghua University":

🎯 Target: Tsinghua University
🌐 Auto-inferred domain: tsinghua.edu.cn

# Tsinghua University Passive Asset Collection Report
> Main domain: tsinghua.edu.cn | Sources: 15
> Total: 285

## 📊 Asset Overview
| Type | Count |
|------|-------|
| subdomain | 270 |
| IP address | 178 |
| port | 2 |
| **Total** | **285** |

## 🚨 Risk Findings
- 🔴 [P1] VPN entry exposed: vpn.tsinghua.edu.cn
- 🔴 [P1] WebVPN remote access exposed: webvpn.tsinghua.edu.cn

---

🔧 Environment Variables

Variable	Type	Description	Required
PASSIVE_API_KEYS	JSON string	Data source API keys (see below)	✅ For some sources
PASSIVE_API_TOKENS	Comma-separated	REST API auth tokens	Optional
PASSIVE_API_KEY	String	Single token fallback	Optional
PASSIVE_DB_PATH	Path	SQLite path (default: data/agent.db)	Optional
PASSIVE_LOG_PATH	Path	Audit log path (default: data/audit.jsonl)	Optional
PASSIVE_PII_SALT	String	PII de-identification salt	Optional
PASSIVE_PII_KEY	String	PII encryption key	Optional

API Key JSON Format

{
  "hunter": ["key1", "key2"],
  "qichacha": {
    "app_key": "your_app_key",
    "secret_key": "your_secret_key"
  },
  "shodan": "your_shodan_key",
  "virustotal": "your_vt_key",
  "zoomeye": "your_zoomeye_key"
}

---

📁 Project Structure

├── cli.py                          ← CLI entry point (one command to rule them all)
├── config.example.json             ← Config template
├── requirements.txt                ← Python dependencies
│
├── passive_agent/                  ← 🎯 Core source
│   ├── main.py                     ← FastAPI app + dashboard API
│   ├── config.py                   ← Config loader (env > config.json)
│   ├── api/                        ← REST API routes
│   ├── collector/                  ← 15 passive data source collectors
│   ├── enumerator/                 ← Subject enumeration engine
│   ├── verifier/                   ← DNS-only verification pipeline
│   ├── compliance/                 ← Compliance guardrail
│   ├── gateway/                    ← Proxy gateway + rate limiter
│   ├── orchestrator/               ← Orchestration scheduler
│   ├── approval/                   ← Approval workflow
│   ├── audit/                      ← Audit logging
│   ├── inventory/                  ← Asset inventory
│   ├── graph/                      ← Knowledge graph (planned)
│   ├── metrics/                    ← Metrics (planned)
│   ├── scheduler/                  ← Daily scheduled tasks
│   ├── storage/                    ← SQLite + JSON persistence
│   ├── common/                     ← Shared components
│   └── static/                     ← Frontend static files
│
├── tests/                          ← Test suite
├── scripts/                        ← CI guard scripts
├── docs/                           ← Design documents
└── data/                           ← Runtime data (gitignored)

Full design docs live in docs/ — system design, sequence diagrams, and class diagrams.

---

🧪 Running Tests

# Full test suite
pytest

# Passive egress guard (CI gate)
pytest tests/test_passive_egress.py -v

# Static guard — ensures no active-scan code enters production
python scripts/guard_passive.py

The passive egress guard and static guard run automatically in CI (.github/workflows/ci.yml) and block any active-scan code from merging.

---

🛡️ Core Principles

1. Purely Passive — Never send a single packet to the target
2. Zero Logs — Never touch the target's logging systems
3. Compliant Egress — Every outbound call must pass compliance_client.check()
4. Fail-Closed — If in doubt, deny; if misconfigured, deny; if no token, deny
5. Auditable — Every operation writes to the audit log (data/audit.jsonl) for compliance review

---

🤝 Contributing

PRs are welcome! Whether it's adding a new data source adapter, improving the dashboard, or fixing a bug — all contributions help make passive recon more powerful.

---

📜 License

MIT

---

Passive Recon — Star ⭐ on GitHub
Made with ❤️ for the OSINT / EASM / CTEM community

---

License & Usage Notice

Source-Available · All Rights Reserved

This project is source-available and all rights are reserved by the author. The code is provided for viewing and evaluation purposes only — access does not grant any right to copy, modify, redistribute, use commercially, or create derivative works. Unauthorized reuse may carry legal risk. Contact the author for explicit written permission before any other use.

本仓库为「源码可查、权利保留」项目（source-available / all-rights-reserved）。代码仅供查看与评估，未授权任何复制、再分发、修改、商用或衍生创作。擅自借用代码存在法律风险；如有需要请先联系作者获取明确书面许可。