Amirhosein-gh98/Gnosis
Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits
This tool helps AI practitioners and researchers evaluate the reliability of responses from large language models (LLMs). By attaching a lightweight 'self-awareness head' to an existing LLM, it predicts a numerical probability of correctness for each generated answer. This allows users to understand how confident the LLM is in its own output, providing a crucial metric for tasks like question answering, content generation, and summarization.
Use this if you need to assess the trustworthiness of an LLM's output and want to automatically flag potentially incorrect answers.
Not ideal if you are looking for a tool to improve the LLM's core answer generation ability itself, as this focuses on evaluating existing outputs.
Stars
32
Forks
9
Language
Python
License
—
Category
Last pushed
Jan 08, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/Amirhosein-gh98/Gnosis"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vectara/hallucination-leaderboard
Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents
PKU-YuanGroup/Hallucination-Attack
Attack to induce LLMs within hallucinations
amir-hameed-mir/Sirraya_LSD_Code
Layer-wise Semantic Dynamics (LSD) is a model-agnostic framework for hallucination detection in...
NishilBalar/Awesome-LVLM-Hallucination
up-to-date curated list of state-of-the-art Large vision language models hallucinations...
intuit/sac3
Official repo for SAC3: Reliable Hallucination Detection in Black-Box Language Models via...