lechmazur/confabulations
Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
This benchmark helps you evaluate how well different large language models (LLMs) avoid making up answers when asked misleading questions based on specific documents. It takes a list of LLMs and a set of challenging questions, then shows you which models are most likely to 'confabulate' or hallucinate, and which tend to simply not respond. An AI developer or researcher building RAG systems would use this to pick the best LLM for their application.
243 stars. No commits in the last 6 months.
Use this if you need to choose an LLM for a Retrieval-Augmented Generation (RAG) system and want to minimize incorrect, made-up answers.
Not ideal if you are looking for an LLM benchmark focused on general accuracy or creative writing rather than specifically confabulation rates.
Stars
243
Forks
9
Language
HTML
License
—
Category
Last pushed
Aug 07, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/lechmazur/confabulations"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
onestardao/WFGY
WFGY: open-source reasoning and debugging infrastructure for RAG and AI agents. Includes the...
KRLabsOrg/verbatim-rag
Hallucination-prevention RAG system with verbatim span extraction. Ensures all generated content...
iMoonLab/Hyper-RAG
"Hyper-RAG: Combating LLM Hallucinations using Hypergraph-Driven Retrieval-Augmented Generation"...
frmoretto/clarity-gate
Stop LLMs from hallucinating your guesses as facts. Clarity Gate is a verification protocol for...
project-miracl/nomiracl
NoMIRACL: A multilingual hallucination evaluation dataset to evaluate LLM robustness in RAG...