M0gician/RaccoonBench
[ACL 2024] Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applications
This tool helps evaluate how vulnerable your LLM-integrated applications are to prompt extraction attacks. It takes your LLM system, along with various attack scenarios and defense mechanisms, and outputs a comprehensive evaluation of its susceptibility. This is for AI/ML security engineers and developers who build and deploy custom GPTs or other LLM-powered applications.
No commits in the last 6 months.
Use this if you need to rigorously test the security of your LLM applications against potential prompt theft and understand the effectiveness of your defenses.
Not ideal if you are looking for an off-the-shelf solution to prevent attacks without needing to run detailed benchmarks.
Stars
14
Forks
—
Language
Python
License
GPL-3.0
Category
Last pushed
May 24, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/M0gician/RaccoonBench"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
gunthercox/chatterbot-corpus
A multilingual dialog corpus
EdinburghNLP/awesome-hallucination-detection
List of papers on hallucination detection in LLMs.
jfainberg/self_dialogue_corpus
The Self-dialogue Corpus - a collection of self-dialogues across music, movies and sports
jkkummerfeld/irc-disentanglement
Dataset and model for disentangling chat on IRC
Tomiinek/MultiWOZ_Evaluation
Unified MultiWOZ evaluation scripts for the context-to-response task.