KazKozDev/system-prompt-benchmark
Test your LLM system prompts against 287 real-world attack vectors including prompt injection, jailbreaks, and data leaks.
When you're building products with Large Language Models (LLMs), this tool helps you automatically test your LLM's core instructions, known as 'system prompts.' You provide your system prompt and select an LLM provider, and the tool runs hundreds of simulated 'attack' scenarios to see how well your prompt defends against things like jailbreaks or data leaks. This is for product managers, AI safety engineers, and anyone deploying LLM-powered applications who needs to ensure their AI behaves as intended.
Use this if you need to rigorously test your LLM system prompts against real-world adversarial inputs before deploying your AI product, ensuring it's robust and secure.
Not ideal if you're testing the entire application pipeline, including user interface elements and complex multi-turn workflows, rather than primarily evaluating the core system prompt's resilience.
Stars
11
Forks
—
Language
Python
License
MIT
Category
Last pushed
Dec 02, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/prompt-engineering/KazKozDev/system-prompt-benchmark"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
protectai/llm-guard
The Security Toolkit for LLM Interactions
MaxMLang/pytector
Easy to use LLM Prompt Injection Detection / Detector Python Package with support for local...
utkusen/promptmap
a security scanner for custom LLM applications
agencyenterprise/PromptInject
PromptInject is a framework that assembles prompts in a modular fashion to provide a...
Resk-Security/Resk-LLM
Resk is a robust Python library designed to enhance security and manage context when...