Skip to content
BranchesTags
 
 

Latest commit

 

History

10 Commits

Folders and files

NameName
Last commit message
Last commit date
.github/workflows
.github/workflows
 
 
.gitignore
.gitignore
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
gen_readme.py
gen_readme.py
 
 

Repository files navigation

Awesome AI Security Tools Awesome

A curated list of public-source, research, and commercial tools for AI security and AI-assisted cybersecurity — autotriage, agent security, AI/ML supply chain, pentest agents, AI SAST, LLM-driven fuzzing, threat intelligence, SOC/SIEM triage, reverse engineering, LLM red-teaming, and more.

License: CC0-1.0 PRs Welcome

Type legend: 🟢 public source / open-source · 🔬 research (paper / benchmark / dataset / framework) · 🟠 commercial with open components · ⚠️ restrictive, non-commercial, or unclear/no license — check before use.

GitHub-hosted entries show live ★ stars and last-commit badges (rendered from the GitHub API by shields.io). Hugging Face model entries show license, access, and artifact metadata. Ordering within a section favors flagship and actively maintained projects.

Contents

Autotriage of Security Findings

AI/LLM tools that triage, deduplicate, prioritize, or validate the output of scanners and finding sources.

  • nuclei-autotriage 🟢⚠️ — Two-stage LLM triage (falsifier + red-team pass) of Nuclei JSONL findings via OpenAI-compatible endpoints (vLLM/Ollama). (CyberOK) — note: restrictive personal/non-commercial EULA, not a permissive OSS license. stars updated
  • seclab-taskflow-agent 🟢 — YAML-driven taskflow agent framework for triaging CodeQL/SAST alerts and filtering false positives. (GitHub Security Lab) stars updated
  • honeyslop 🟢 — Code-canary decoys to triage AI-hallucinated ("slop") vulnerability reports flooding bug-bounty programs. stars updated
  • nano-analyzer 🟢🔬 — Minimal three-stage LLM pipeline (context → scan → skeptical triage) for zero-day discovery in C/C++. (AISLE) stars updated
  • SigmaOptimizer 🟢 — Generates, tests, and refines Sigma rules from real logs with false-positive checking. stars updated
  • ai-soc-triage-assistant 🟢⚠️ — SOC alert triage assistant with prompt-injection guardrails, output validation, and MITRE ATT&CK mapping. stars updated

See also: OpenAI's Aardvark / Codex Security research previews — public references exist, but there is no standalone installable repo to badge here.

AI Agent & Coding-Agent Security

Securing the AI agents themselves — auditing coding agents (Claude Code, Codex, OpenClaw), scanning skills / plugins / MCP manifests, and governance for agentic development. A fast-moving 2026 category, split below by role.

Scanners & Auditors

Frameworks, Rule Standards & Benchmarks

Runtime Protection & Enforcement

  • onecli 🟢 — Credential gateway and encrypted vault for AI agents; injects real API credentials at the gateway so agents only see placeholder keys. stars updated
  • microsandbox 🟢 — Local-first, microVM-backed programmable sandboxes for AI agents with SDKs, CLI, MCP support, and rootless hardware isolation. stars updated
  • agentguard 🟢 — Real-time security layer for coding agents: hooks scan every new skill, block dangerous actions before execution, run daily posture patrols, and track which skill triggered each action (incl. Web3-specific checks). stars updated
  • defenseclaw 🟠 — Enforcement and evidence layer for agentic deployments: static CodeGuard checks, sandboxing, registry ingestion with SSRF guards, and audit/observability. (Cisco AI Defense) stars updated
  • AgentFence 🟢 — Runtime-oriented test harness for deployed agents, probing prompt injection, secret leakage, and system-instruction exposure. Early-stage project. stars updated
  • clawsec 🟢⚠️ — Security skill suite for OpenClaw-family agents; AGPL-3.0 licensed. (Prompt Security) stars updated

AI/ML Supply Chain & Model Security

Tools for securing model artifacts, serialized ML files, AI/ML supply-chain surfaces, and malicious-package detection datasets/benchmarks.

  • Fraim 🟢 — Framework for AI-powered security workflows including LLM SAST and IaC analysis with SARIF/HTML output. stars updated
  • Adversarial Robustness Toolbox (ART) 🟢 — Flagship machine-learning security library for evaluating and defending models against evasion, poisoning, extraction, and inference attacks across major ML frameworks. (LF AI & Data / IBM) stars updated
  • Foolbox 🟢 — Classic Python toolbox for generating adversarial examples and benchmarking robustness of PyTorch, TensorFlow, and JAX models. stars updated
  • modelscan 🟢 — Scans ML model files for unsafe serialization patterns and embedded code, with a focus on model serialization attacks. (Protect AI) stars updated
  • Fickling 🟢 — Python pickle decompiler, rewriter, and static analyzer for inspecting and detecting malicious pickle/PyTorch payloads. (Trail of Bits) stars updated
  • picklescan 🟢 — Lightweight CLI/library for detecting suspicious Python pickle operations in ML and model artifacts. stars updated
  • AIsbom 🟢 — AI software bill of materials tooling for AI/ML supply-chain inventory and provenance metadata. stars updated
  • model-provenance-kit 🟢 — Toolkit for model-family provenance and fingerprinting across model weights, tokenizers, and architecture signals. (Cisco AI Defense) stars updated
  • pickle-fuzzer 🟢 — Structure-aware fuzzer for pickle scanners, useful for hardening tools such as modelscan, Fickling, and picklescan. (Cisco AI Defense) stars updated
  • Medusa 🟢⚠️ — AI-first security scanner for AI/ML repos, agents, and MCP surfaces; AGPL-3.0 licensed. (Pantheon Security) stars updated
  • PrivacyRaven 🟢🔬 — Privacy-testing library for deep-learning systems, covering model extraction and membership-inference style attacks. (Trail of Bits) — note: archived/hiatus project, but still a useful reference implementation. stars updated
  • gym-malware 🟢🔬 — OpenAI Gym environment for reinforcement-learning agents that mutate PE malware to evade static ML malware detectors. stars updated
  • deep-pwning 🟢🔬 — Historical "Metasploit for machine learning" framework for experimenting with adversarial robustness of ML models. stars updated
  • open-malicious-code-benchmark 🟢🔬 — OMCBench benchmark suite for malicious-code/package detection: labeled Python and JavaScript package archives, common runners, and published precision/recall/F1 metrics. (False Positive Community) — note: evaluates an unreleased commercial ML detector (MOLOT / PT Application Inspector) alongside open-source baselines. stars updated
  • malicious-software-packages-dataset 🟢🔬 — Human-vetted dataset of malicious software packages across npm, PyPI, IDE extensions, and AI Skills, useful for detector training and evaluation. (Datadog Security Labs) — note: contains real malware samples; Datadog notes selection bias because many samples were identified by GuardDog. stars updated
  • GuardDog 🟢 — CLI for detecting malicious PyPI, npm, Go, RubyGems, GitHub Actions, and VSCode extension packages using Semgrep rules and package-metadata heuristics. (Datadog) stars updated
  • package-analysis 🟢🔬 — Sandboxed static/dynamic analysis pipeline for open-source packages, capturing filesystem, process, and network behavior and publishing data for malicious-package research. (OpenSSF) stars updated
  • malicious-code-ruleset 🟢 — Focused Semgrep ruleset for malicious-code patterns such as dynamic execution and obfuscation, used as an OMCBench baseline. (Apiiro) stars updated
  • pypi_malregistry 🔬⚠️ — ASE'23 / USENIX Security'26 malicious-PyPI dataset with more than 10k malicious package versions. — note: no LICENSE file found and the repository contains malware samples; handle in an isolated environment. stars updated

Pentest & Red-Team Agents

Autonomous and semi-autonomous AI agents for penetration testing, exploitation, and attack simulation.

  • PentestGPT 🟢🔬 — The original USENIX'24 LLM pentest agent; re-released as an autonomous pipeline with strong benchmark results. stars updated
  • PentAGI 🟢 — Fully autonomous multi-agent pentest framework with Docker sandboxing. (VXControl) stars updated
  • CAI – Cybersecurity AI 🟢🟠 — Modular, bug-bounty-ready agent framework supporting 300+ LLM models. MIT for research; separate commercial license for production/on-prem. (Alias Robotics) stars updated
  • Strix 🟢 — Autonomous "AI hackers" that dynamically run code and validate vulnerabilities with PoCs (Apache-2.0). stars updated
  • hackingBuddyGPT 🟢🔬 — Minimal (~50 LOC) research framework for LLM-driven Linux priv-esc and web pentesting (FSE'23). stars updated
  • Nebula 🟢🟠 — AI pentesting CLI assistant with local-LLM support (Llama-3.1, Mistral, DeepSeek). stars updated
  • HexStrike-AI 🟢 — MCP server exposing 150+ security tools (nmap, gobuster, nuclei, …) to AI agents (MIT). stars updated
  • Burp Suite MCP Server 🟢⚠️ — Official Burp Suite extension exposing Burp to AI clients through MCP. (PortSwigger) — note: GPL-3.0 licensed. stars updated
  • pentest-ai 🟢 — Offensive-security MCP server with 200+ wrapped tools, specialist agents, and OWASP-oriented probes for authorized testing. stars updated
  • pentest-ai-agents 🟢 — Collection of Claude Code offensive-security subagents for authorized penetration-testing research. stars updated
  • Shannon 🟢🟠⚠️ — White-box autonomous AI pentester with strong XBOW-benchmark results. Shannon Lite is AGPL-3.0; Shannon Pro is commercial. stars updated
  • AIDA 🟢⚠️ — Model-agnostic autonomous pentest agent running inside an isolated Docker environment; AGPL-3.0 licensed. stars updated
  • HackSynth 🟢🔬⚠️ — Planner/summarizer LLM-agent framework for autonomous penetration testing and benchmark evaluation; AGPL-3.0 licensed. stars updated
  • VulnBot 🟢🔬 — Multi-agent collaborative penetration-testing framework with RAG support. stars updated
  • PentestAgent 🟢 — Black-box AI pentest framework with MCP, multi-agent spawning, and persistent sessions. stars updated
  • cyber-security-llm-agents 🟢⚠️ — AutoGen-based agents for cybersecurity tasks (shown at RSAC 2024). (NVISO) stars updated
  • Pentest-Swarm-AI 🟢 — Swarm-intelligence multi-agent pentest with stigmergic blackboard coordination (Go). stars updated
  • hackGPT 🟢⚠️ — LLM offensive-security toolkit. stars updated

AI-Powered Recon & Narrow ML Tools

Hyper-specific AI/ML tools for a single offensive-security, recon, or detection step — the subwiz/eyeballer pattern rather than broad autonomous agents. 🅐 = self-contained trained model or learned model/pattern engine; 🅑 = LLM wrapper that calls an external API.

Subdomain & DNS Prediction

  • subwiz 🟢 — 🅐 Lightweight nanoGPT model that predicts resolvable subdomains via beam search; model weights are published on Hugging Face. (Hadrian Security) stars updated
  • regulator 🟢⚠️ — 🅐 Learns and ranks regex-like naming patterns from known subdomains to generate likely new candidates. — note: no LICENSE file found; treat as source-available until clarified. stars updated

Recon Screenshot Triage

  • eyeballer 🟢⚠️ — 🅐 Convolutional neural network that classifies pentest/recon screenshots (login pages, webapps, old-looking sites, parked domains, and custom 404s) for attack-surface triage. (Bishop Fox) — note: GPL-3.0 licensed. stars updated

Software / Tech Fingerprinting

  • GyoiThon 🟢🔬 — 🅐 Machine-learning-assisted web intelligence tool that fingerprints products, versions, CVEs, login pages, debug messages, and related web-server signals from HTTP responses. — note: older project; Apache-2.0 licensed, but maintenance appears low. stars updated
  • http-fingerprint-transformer 🔬⚠️ — 🅐 Transformer-encoded HTTP-header fingerprinting experiment for classifying web-server software and versions. — note: thesis artifact; code is GPL-3.0 and data is CC BY-SA 4.0. stars updated

AI-Assisted Fuzzing

  • ffufai 🟢⚠️ — 🅑 AI wrapper around the ffuf web fuzzer that suggests file extensions and paths from the target URL and headers using OpenAI or Anthropic models. (Joseph Thacker) — note: requires an LLM API key; README states MIT but no LICENSE file was found. stars updated

Password / Credential ML

  • PassGPT 🔬⚠️ — 🅐 GPT-style password model trained on leaked passwords for research on password generation and strength estimation. (Rando et al.) license: CC BY-NC-4.0 · access: open 10-char model; 16-char variant gated · artifacts: PyTorch/Safetensors. Research-only / non-commercial use; related code: javirandor/passgpt.
  • PassGAN 🔬 — 🅐 WGAN that learns password distributions from leaks to generate guesses; historical reference implementation of the PassGAN paper (MIT). stars updated
  • neural_network_cracking 🔬 — 🅐 RNN password-guessing model from Fast, Lean, and Accurate: Modeling Password Guessability Using Neural Networks (USENIX Security 2016); Apache-2.0 licensed. (CMU CUPS Lab) stars updated

Phishing Detection (Visual / URL)

  • phishing-url-detection 🟢 — 🅐 Packaged URL phishing classifier with ONNX and pickle artifacts. license: MIT · access: open · artifacts: ONNX, pickle. Model card recommends ONNX over pickle for safer inference.
  • PhishIntention 🔬 — 🅐 Deep-vision phishing detector that infers both brand intention and credential-taking intention from webpage appearance and dynamics (USENIX Security 2022). — note: CC0-1.0 licensed. stars updated
  • VisualPhishNet 🔬⚠️ — 🅐 Triplet CNN for zero-day phishing detection by visual similarity to trusted websites (ACM CCS 2020). (CISPA) — note: no LICENSE file found; dataset access is research-request based. stars updated

ML-Generated Detection Rules

  • yaraml_rules 🟢 — 🅐 Trains scikit-learn classifiers on malware/benign corpora and compiles the learned model into deployable YARA rules (Apache-2.0). (Sophos) stars updated

Defensive Trained-Model Detectors

  • DeepSQLi 🟢⚠️ — 🅐 Deep-learning SQL-injection detector with dataset, trained models, and a Flask Prediction API for GatewayD IDS/IPS integration. (GatewayD) — note: AGPL-3.0 licensed; defensive detector rather than offensive generator. stars updated
  • deepsecrets 🟢 — Semantic secrets scanner using lexing/parsing, entropy checks, and hashed-known-secret matching across 500+ languages. — note: useful narrow detector, but not a trained ML model. stars updated

AI-Powered SAST & Secure Code Review

Static analysis and secure code review enhanced with LLMs.

  • Vulnhuntr 🟢 — Zero-shot vulnerability discovery in Python repos via LLM call-chain analysis; credited with a 0-day RCE in Ragflow. (Protect AI) stars updated
  • deepsec 🟢 — Agent-powered security harness for scanning large codebases with coding agents, resumable parallel runs, custom matchers, and optional revalidation. (Vercel Labs) stars updated
  • claude-code-security-review 🟠 — Official Claude-based semantic SAST GitHub Action that reviews PR diffs. (Anthropic) stars updated
  • IRIS 🟢🔬 — Neurosymbolic SAST combining LLMs with CodeQL for Java vulnerability detection (MIT). stars updated
  • sast-skills 🟢 — Agent skills that turn AI coding assistants into a multi-agent SAST scanner. stars updated
  • llm-sast-scanner 🟢 — SAST skill for AI coding agents with structured source-to-sink analysis across 34 vulnerability classes. License: MIT stated in README. stars updated
  • sast-ai-workflow 🟢 — LangGraph workflow for reviewing static-analysis findings, reducing false positives, and producing vulnerability review output. (Red Hat Ecosystem AppEng) stars updated
  • xvulnhuntr 🟢 — Archived fork of Vulnhuntr extending support to C#, Java, and Go. (Compass Security) stars updated
  • llm-security-scanner 🟢⚠️ — LLM-powered code scanner that opens GitHub issues for findings. stars updated
  • vulnhuntr-mod 🟢 — Modified Vulnhuntr with Qwen/Hunyuan support and Chinese-language prompts. stars updated

LLM-Driven Fuzzing

Two families: (a) LLMs generating harnesses/targets for traditional fuzzing, and (b) fuzzing the LLM itself.

Harness / target generation

  • oss-fuzz-gen 🟢 — LLM-driven fuzz-harness generation for OSS-Fuzz; reported 26 real vulnerabilities (incl. CVE-2024-9143 in OpenSSL). (Google) stars updated
  • PromptFuzz 🟢🔬⚠️ — LLM-mutated prompts to generate fuzz drivers for C/C++ libraries (Rust). stars updated
  • Fuzz4All 🟢🔬 — "Universal" LLM-based fuzzer across compilers/languages (ICSE 2024). stars updated
  • ChatAFL 🟢🔬 — LLM-guided protocol fuzzing extending AFLNet (NDSS'24). stars updated
  • TitanFuzz 🟢🔬⚠️ — First LLM-based fuzzer for PyTorch/TensorFlow (ISSTA'23). stars updated

Fuzzing the LLM

  • LLMFuzzer 🟢 — First open-source fuzzing framework for LLM API integrations. stars updated
  • ps-fuzz 🟠 — System-prompt hardening fuzzer; 16 attacks × 16 providers. (Prompt Security) stars updated
  • FuzzyAI 🟠 — Automated LLM fuzzer for jailbreaks/prompt injection. (CyberArk) stars updated
  • spikee 🟢 — Prompt-injection evaluation and exploitation kit with dataset generation, Burp integration, and pluggable judges. (ReversecLabs / WithSecure) stars updated
  • promptmap 🟢⚠️ — Prompt-injection scanner for custom LLM applications in white-box and black-box modes; GPL-3.0 licensed. stars updated
  • ai-prompt-fuzzer 🟢 — Burp Suite extension fuzzing GenAI/LLM prompts. (PortSwigger) stars updated

Threat Intelligence

AI/LLM tooling for CTI gathering, IOC/TTP extraction, and analysis.

  • trs 🟢 — LLM + ChromaDB tool to summarize threat reports and extract MITRE TTPs and IOCs. stars updated
  • TI-Mindmap-GPT 🟢 — Streamlit app: AI summaries, mindmaps, IOC/TTP extraction, and ATT&CK Navigator layers. stars updated
  • aiocrioc 🟢 — LLM + OCR IOC extraction (pulls IOCs from images/PDFs). stars updated
  • ThreatIngestor 🟢 — Extracts/aggregates IOCs from feeds; integrates with MISP/ThreatKB (pairs well with LLM post-processing). stars updated
  • IATelligence 🟢 — Explains imported Windows APIs in PE files via GPT and maps to MITRE ATT&CK. stars updated
  • MCP_Security 🟢⚠️ — MCP server (ORKL) for querying the ORKL threat-intel API. stars updated
  • threat-intelligence-cti-analysis 🟢 — NLP/LLM pipeline for IOC extraction, MITRE ATT&CK mapping, and knowledge-graph generation from unstructured CTI. stars updated

Log Analysis / SIEM / SOC Triage

AI agents for SOC alert triage, investigation, and incident response.

  • AI-SOC-Agent 🟢 — Black Hat 2025 MCP server exposing security-investigation tools (ELK, IRIS). stars updated
  • soctalk 🟢 — LangGraph SOC automation agent with MCP integrations for Wazuh, Cortex, TheHive, and MISP plus mock-agent test lab. stars updated
  • Vigil SOC 🟢 — Open-source AI SOC with readable Python agents, Markdown playbooks, and MCP integrations for triage, investigation, hunting, response, reporting, and forensics. (Vigil SOC) stars updated
  • agentic-soc-platform 🟢 — Agentic SOC platform (LangGraph/Dify) with local-LLM support. stars updated
  • SigmAIQ 🟢⚠️ — pySigma wrapper and LangChain toolkit for automatic Sigma rule creation and translation; LGPL-2.1 licensed. (AttackIQ) stars updated
  • SOCGPT 🟢 — LLM log summarization, severity triage, MITRE mapping, and Q&A. stars updated
  • AttackGen 🟢 — LLM-driven incident-response scenario generator using MITRE ATT&CK + ATLAS. stars updated

Reverse Engineering

LLM-assisted binary analysis and traffic inspection.

  • Gepetto 🟢 — IDA Pro plugin: GPT adds comments and meaningful variable names. stars updated
  • ida-pro-mcp 🟢 — MCP bridge for IDA Pro exposing decompile, disassemble, xref, rename, and debugging workflows to LLM clients. stars updated
  • GhidraMCP 🟢 — MCP server exposing Ghidra reverse-engineering ops to any MCP-capable LLM. stars updated
  • ReVa 🟢 — Ghidra-focused reverse-engineering assistant with MCP support, Claude Skills integration, and long-form analysis workflows. stars updated
  • GhidrAssistMCP 🟢 — Native Ghidra MCP extension with broad tool coverage, headless support, and security-sensitive tool gating. stars updated
  • ghidra-mcp 🟢 — Ghidra MCP server with large tool coverage, GUI plugin, headless server, and lazy tool loading. stars updated
  • GhidrOllama 🟢⚠️ — Ghidra script using the Ollama API for function analysis/renaming. stars updated
  • GhidraGPT 🟢 — Ghidra plugin that integrates LLMs for automated code refactoring and analysis. stars updated
  • LLM4Decompile 🟢🔬⚠️ — Research project for binary-to-C decompilation with LLMs; code is MIT, but model weights use a more restrictive license. stars updated
  • x64dbg_mcp 🟢 — MCP server exposing x64dbg debugging and reverse-engineering operations to AI clients. stars updated
  • binaryninja-mcp 🟢 — MCP server for Binary Ninja-assisted reverse engineering. stars updated
  • OGhidra 🟢 — Natural-language Ghidra analysis via Ollama. (Lawrence Livermore National Lab) stars updated
  • ghidra_tools (G-3PO) 🟢 — Ghidra plugin for AI-assisted decompiled-code analysis. (Tenable) stars updated
  • gpt-wpre 🔬 — Whole-program reverse engineering with GPT-3. stars updated
  • burpgpt 🟢 — Burp Suite extension integrating GPT for passive scanning. stars updated
  • Burp-extension-for-GPT 🟢 — Burp extension to analyze HTTP traffic with GPT. (Tenable) stars updated

LLM Red-Teaming & Guardrails

Tools for attacking and defending LLM applications themselves.

Scanners, Evals & Guardrails

  • garak 🟢 — The LLM vulnerability scanner — probes for prompt injection, jailbreaks, data leakage, and more. (NVIDIA) stars updated
  • PyRIT 🟢 — Python Risk Identification Tool; battle-tested across 100+ GenAI red-team operations. (Microsoft) stars updated
  • promptfoo 🟢 — LLM eval + red-teaming/pentesting CLI with 50+ attack plugins (MIT). Note: OpenAI announced an acquisition agreement in March 2026; remains MIT-licensed — track governance. stars updated
  • Augustus 🟢 — Single-binary LLM security testing framework for prompt injection, jailbreaks, and adversarial attacks across many providers. (Praetorian) stars updated
  • agentic_security 🟢 — Agentic LLM vulnerability scanner and AI red-team kit for jailbreaks, prompt injection, fuzzing, and API stress testing. stars updated
  • HiveTrace Red 🟢 — Early-stage LLM red-teaming framework with 80+ attack templates, async evaluation pipelines, WildGuard evaluators, multi-provider support, and HTML reporting. — note: young project with limited independent adoption signal. stars updated
  • DeepTeam 🟢 — Open-source framework for red-teaming LLMs and LLM systems across jailbreaks, prompt injection, data leakage, and safety risks. stars updated
  • Moonshot 🟢 — Modular tool for benchmarking, red-teaming, and evaluating LLM applications with custom connectors and recipes. (AI Verify Foundation) stars updated
  • Guardrails AI 🟢 — Python framework for adding input/output guards, validators, structured-output controls, and Guardrails Hub checks to LLM applications. (Guardrails AI) stars updated
  • Giskard 🟢 — Open-source evaluation, testing, and red-teaming framework for LLM agents, including agent vulnerability scanning and RAG evaluation workflows. (Giskard AI) stars updated
  • LangKit 🟢 — LLM monitoring toolkit extracting safety/security signals such as jailbreak similarity, prompt-injection similarity, hallucination checks, PII patterns, toxicity, and refusal metrics. (WhyLabs) stars updated
  • LLM Guard 🟢 — Suite of input/output scanners (PII, prompt injection, etc.). (Protect AI) stars updated
  • Rebuff 🟢 — Archived prompt-injection detector (heuristics + LLM + vector DB + canary tokens). (Protect AI) stars updated
  • NeMo Guardrails 🟢 — Programmable guardrails (input/output/dialog/retrieval rails) for LLM apps. (NVIDIA) stars updated
  • PurpleLlama 🟢 — Llama Guard classifiers, CodeShield, and CyberSecEval. (Meta) stars updated
  • LLAMATOR 🟢⚠️ — Red-teaming framework for chatbots and GenAI systems; CC BY-NC-SA 4.0 licensed. stars updated
  • Vigil 🟢🔬 — Library/REST API to scan prompts and responses for prompt injection. stars updated
  • Counterfit 🟢 — ML/AI penetration-testing automation tool. (Microsoft) stars updated
  • AI-Red-Teaming-Playground-Labs 🟢 — CTFd-based AI red-team training challenges. (Microsoft) stars updated
  • EasyJailbreak 🟢🔬 — Framework for building and testing adversarial jailbreak prompts. stars updated
  • TextAttack 🟢🔬 — Python framework for adversarial attacks, data augmentation, and training for NLP models; useful for robustness testing beyond chat-only LLM scanners. stars updated
  • GPTFuzz 🟢🔬 — Research framework for red-teaming LLMs with auto-generated jailbreak prompts. stars updated
  • HarmBench 🟢🔬 — ICML 2024 standardized evaluation framework for automated red-teaming and robust-refusal benchmarking. (Center for AI Safety) stars updated
  • llm-attacks (GCG) 🟢🔬 — Canonical Greedy Coordinate Gradient adversarial-suffix attack implementation for transferable attacks on aligned language models. stars updated
  • nanoGCG 🟢 — Fast, lightweight PyTorch implementation of the GCG adversarial-suffix algorithm. stars updated
  • JailbreakBench 🟢🔬 — NeurIPS 2024 open robustness benchmark and leaderboard for generating and defending against LLM jailbreaks. stars updated
  • Open-Prompt-Injection 🟢🔬 — Open-source toolkit and benchmark for implementing and evaluating prompt-injection attacks, defenses, and LLM-integrated applications. stars updated
  • Whistleblower 🟢⚠️ — Offensive testing tool for inferring system prompts and discovering capabilities of LLM applications exposed through APIs. (Repello AI) — note: no LICENSE file found. stars updated
  • LLMmap 🟢🔬 — Minimal-query fingerprinting tool for identifying LLMs from behavioral traces, with a pretrained open-set inference model. stars updated
  • llm-security 🔬 — Original PoC for indirect prompt-injection attacks. stars updated
  • JailbreakLLMs 🔬⚠️ — Research dataset of 6,387 ChatGPT prompts, including in-the-wild jailbreak prompts from Reddit, Discord, websites, and open datasets. stars updated
  • Do-Not-Answer 🟢🔬 — Dataset for evaluating LLM safeguards on unsafe or policy-sensitive prompts. stars updated
  • prompt-injection-defenses 🟢⚠️ — Curated catalog of practical defenses against prompt injection. stars updated

Prompt-Injection Classifier Models

  • Wolf Defender Prompt Injection 🟢 — Hugging Face text-classification model for prompt-injection detection in agents, chatbots, and CI workflows. (Patronus Studio / Casdo Labs) license: Apache-2.0 · access: open · artifacts: Safetensors, ONNX.
  • DeBERTa v3 Prompt Injection v2 🟢 — Apache-licensed prompt-injection classifier usable via Transformers pipelines and ONNX. (Protect AI) license: Apache-2.0 · access: open · artifacts: Safetensors, ONNX.
  • PromptGuard 🟢⚠️ — ModernBERT-based prompt-injection and jailbreak classifier. (CodeIntegrity AI) license: Apache-2.0 · access: gated auto · artifacts: Safetensors.
  • Prompt Guard 86M 🟠⚠️ — Meta prompt-injection and jailbreak classifier from the Llama Guard family. (Meta) license: Llama 3.1 · access: gated manual · artifacts: Safetensors.
  • prompt-injection-sentinel 🔬⚠️ — ModernBERT-large classifier for prompt-injection and jailbreak detection. (Qualifire) license: other · access: gated auto · artifacts: Safetensors.

Specialty Security LLMs

  • SecGPT 🟢 — Open cybersecurity-tuned LLM family for vulnerability analysis, log/traffic investigation, anomaly detection, attack/defense reasoning, command analysis, and security Q&A. (Clouditera) stars updated

LLM Honeypots & Deception

Honeypots and deception that use LLMs to simulate convincing systems.

  • Beelzebub 🟢 — Low-code honeypot using LLMs to simulate SSH/HTTP/MCP services (Go). stars updated
  • DECEIVE 🟢🔬 — Proof-of-concept LLM-powered SSH honeypot that evaluates sessions as benign, suspicious, or malicious. (Splunk) stars updated
  • TRAP 🟢🔬 — Research code for Targeted Random Adversarial Prompt honeypots that identify black-box LLM usage through model-specific prompt suffixes (ACL 2024 Findings). stars updated
  • shelLM 🟢🔬 — LLM-powered SSH honeypot (paper "LLM in the Shell"). stars updated
  • VelLMes 🟢🔬 — Multi-protocol LLM honeypot framework (successor to shelLM). stars updated
  • llm-honeypot 🔬⚠️ — Cowrie SSH honeypot extended with prompt-injection traps to detect LLM hacker agents. (Palisade Research) stars updated

CTF / Exploit / Bug-Bounty Agents & Benchmarks

Offensive agents and the benchmarks used to evaluate them.

  • SWE-agent (EnIGMA) 🟢🔬 — EnIGMA offensive-CTF mode; SOTA on NYU CTF, InterCode-CTF, and Cybench (v0.7 branch). stars updated
  • Cybench 🔬 — 40 professional CTF tasks across 4 competitions; widely used by AI safety institutes. stars updated
  • NYU CTF Bench 🔬 — Dockerized CSAW CTF challenges for LLM-agent evaluation. stars updated
  • CTFTiny 🔬⚠️ — Lightweight CTF benchmark from the NYU LLM CTF group; GPL-2.0 licensed. stars updated
  • InterCode 🔬 — Interactive-coding benchmark incl. InterCode-CTF. stars updated
  • inspect_evals 🟢🔬 — Maintained Inspect AI evaluation suite containing multiple cyber benchmarks and tasks. (UK AI Security Institute) stars updated
  • BountyBench 🔬 — 25 real systems / 40 bug bounties for Detect-Exploit-Patch evaluation. stars updated
  • Cyber-Zero 🔬 — Trains cybersecurity agents without runtime; ships an EnIGMA+ scaffold. (Amazon Science) stars updated
  • ExploitBench 🔬 — Measures AI-agent progress on V8/Chromium exploit ladders. stars updated
  • AI Goat 🟢🔬⚠️ — Vulnerable-by-design local LLM CTF for learning prompt injection, insecure output handling, data leakage, excessive agency, and related LLM app risks. — note: GPL-2.0 licensed. stars updated
  • Damn Vulnerable LLM Agent 🟢🔬 — Deliberately vulnerable LangChain ReAct agent for practicing prompt-injection and Thought/Action/Observation injection attacks. (ReversecLabs / WithSecure) stars updated
  • claude-bug-bounty 🟢 — Claude Code plugin orchestrating recon → vuln classes → reporting. stars updated
  • Bug-Bounty-Agents 🟢 — 43 AI agent personas for Claude Code / Copilot / Cursor across the bug-bounty lifecycle. stars updated
  • ai-exploits 🟢 — Real-world AI/ML exploits (Metasploit modules + Nuclei templates) for MLflow, Ray, H2O. (Protect AI) stars updated

Cloud / IaC / DFIR / OSINT / Phishing

AI tooling for cloud/IaC security, digital forensics, OSINT, and phishing detection.

  • EscalateGPT 🟢 — GPT-based discovery of privilege-escalation paths in AWS IAM policies. (Tenable) stars updated
  • MemoryInvestigator 🔬 — Volatility 3 + LLM + RAG for memory-forensic triage. stars updated
  • Volatility-MCP-Server 🟢 — MCP exposing Volatility 3 plugins for natural-language memory forensics. stars updated
  • llm_osint 🟢🔬 — Proof-of-concept LLM OSINT framework using knowledge and web agents for internet research workflows. stars updated
  • ai_osint 🟢 — Curated AI-OSINT dorks, queries, and techniques for discovering exposed LLM and AI infrastructure. stars updated
  • PhishLLM 🔬⚠️ — Reference-less phishing detection via LLM brand recognition (USENIX'24). stars updated
  • mcp-dnstwist 🟢 — MCP server for dnstwist DNS fuzzing to support typosquatting, phishing, and lookalike-domain analysis. stars updated
  • osintgpt 🟢⚠️ — OpenAI embeddings + Qdrant over OSINT corpora. stars updated
  • gpt-osint 🟢 — Web-based GPT-4 OSINT tool over social-media dumps and CSVs. stars updated

Related Awesome Lists

Contributing

Contributions are welcome! Open a PR adding entries in the format below, keeping each section sorted by relevance/maintenance.

- **[name](repo-url)** 🟢/🔬/🟠/⚠️ — One-line description. *(maintainer/org)* <stars badge> <last-commit badge>
  - **Sources:** [upstream A](url) · [upstream B](url)       # optional — projects this is built on
  - **Related:** [sibling tool](url) · [related project](url) # optional — peers / forks / successors

Badges use the dynamic shields.io GitHub endpoints, so they update automatically:

[![stars](https://img.shields.io/github/stars/OWNER/REPO?style=flat-square&label=★)](https://github.com/OWNER/REPO)
[![updated](https://img.shields.io/github/last-commit/OWNER/REPO?style=flat-square&label=updated)](https://github.com/OWNER/REPO)

Guidelines: link the canonical upstream repo (not a fork); verify the URL resolves; tag the correct type and add ⚠️ for non-permissive, non-commercial, or unclear/no-license projects; prefer real, installable projects over blog-only references.

For Hugging Face model entries, include the model id, license, access status (open/gated), and artifact formats (for example Safetensors or ONNX).

License

To the extent possible under law, the contributors have waived all copyright and related rights to this list (CC0-1.0). Linked projects retain their own licenses — check each before use.

About

A curated list of public-source, research, and commercial tools for AI security and AI-assisted cybersecurity — autotriage, agent security, AI/ML supply chain, pentest agents, AI SAST, LLM-driven fuzzing, threat intelligence, SOC/SIEM triage, reverse engineering, LLM red-teaming, and more.

Resources

Readme
License

Contributing

Contributing
Activity
Custom properties

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages