tigerchen52/query_level_uncertainty
query-level uncertainty in LLMs
This project helps operations engineers and developers managing AI applications to quickly assess how confident a large language model (LLM) is about a user's query, before generating an answer. It takes a user's question or prompt as input and outputs a confidence score, allowing for faster decision-making on whether to use additional tools like a RAG system or a more complex model. This is ideal for those who need to manage the cost and latency of LLM-powered systems.
Use this if you need to quickly determine the reliability of an LLM's understanding of a query without waiting for a full answer, helping you decide if you need to escalate to more robust, but slower, solutions like RAG or a more powerful model.
Not ideal if your primary concern is evaluating the quality or accuracy of a generated answer itself, rather than the model's initial confidence in understanding the query.
Stars
9
Forks
3
Language
Python
License
MIT
Category
Last pushed
Mar 27, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/tigerchen52/query_level_uncertainty"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Higher-rated alternatives
cvs-health/uqlm
UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM...
PRIME-RL/TTRL
[NeurIPS 2025] TTRL: Test-Time Reinforcement Learning
sapientinc/HRM
Hierarchical Reasoning Model Official Release
reasoning-survey/Awesome-Reasoning-Foundation-Models
✨✨Latest Papers and Benchmarks in Reasoning with Foundation Models
HKUDS/LightReasoner
"LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?"