PKU-YuanGroup/Hallucination-Attack
Attack to induce LLMs within hallucinations
This project helps evaluate how easily large language models (LLMs) can be tricked into generating false information or 'hallucinations.' It takes a standard LLM and applies specially crafted, often nonsensical prompts to see if the model can be made to produce fake facts or news. This is useful for AI safety researchers, red teamers, and anyone responsible for assessing the reliability and potential risks of LLMs before deployment.
164 stars. No commits in the last 6 months.
Use this if you need to rigorously test an LLM's susceptibility to generating false or misleading content when given unusual or adversarial inputs.
Not ideal if you are looking to improve the factual accuracy of an LLM or fine-tune it for a specific task.
Stars
164
Forks
21
Language
Python
License
MIT
Category
Last pushed
May 17, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/PKU-YuanGroup/Hallucination-Attack"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vectara/hallucination-leaderboard
Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents
amir-hameed-mir/Sirraya_LSD_Code
Layer-wise Semantic Dynamics (LSD) is a model-agnostic framework for hallucination detection in...
NishilBalar/Awesome-LVLM-Hallucination
up-to-date curated list of state-of-the-art Large vision language models hallucinations...
intuit/sac3
Official repo for SAC3: Reliable Hallucination Detection in Black-Box Language Models via...
HillZhang1999/llm-hallucination-survey
Reading list of hallucination in LLMs. Check out our new survey paper: "Siren’s Song in the AI...