aivillage/llm_verification
LLM prompt attacks for hacker CTFs via CTFd.
This plugin helps security event organizers create challenges for participants to test and "attack" Large Language Models (LLMs). Organizers can define challenges where participants craft prompts to make an LLM generate specific, often 'cheeky' or subverting, responses. The organizers then manually review these submitted prompts and their LLM outputs, assigning points based on how effectively the participant's prompt subverted the model.
No commits in the last 6 months.
Use this if you are organizing a Capture The Flag (CTF) or similar cybersecurity competition and want to include challenges focused on red-teaming or subverting AI language models.
Not ideal if you are looking for an automated system to evaluate LLM outputs without human intervention, or if your competition does not use the CTFd platform.
Stars
15
Forks
1
Language
Python
License
—
Category
Last pushed
Dec 17, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/aivillage/llm_verification"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
GreyDGL/PentestGPT
Automated Penetration Testing Agentic Framework Powered by Large Language Models
berylliumsec/nebula
AI-powered penetration testing assistant for automating recon, note-taking, and vulnerability analysis.
ipa-lab/hackingBuddyGPT
Helping Ethical Hackers use LLMs in 50 Lines of Code or less..
MorDavid/BruteForceAI
Advanced LLM-powered brute-force tool combining AI intelligence with automated login attacks
mbrg/power-pwn
An offensive/defense security toolset for discovery, recon and ethical assessment of AI Agents