intuit-ai-research/DCR-consistency
DCR-Consistency: Divide-Conquer-Reasoning for Consistency Evaluation and Improvement of Large Language Models
This tool helps evaluate and improve the accuracy of text generated by large language models (LLMs) by detecting and fixing "hallucinations" or inconsistencies. It takes a reference text (ground truth) and a candidate text from an LLM, then outputs a consistency score, specific reasons for inconsistencies, and an improved version of the candidate text. This is for anyone who uses or develops applications powered by LLMs and needs to ensure their outputs are factual and reliable.
No commits in the last 6 months.
Use this if you need to objectively measure the factual consistency of LLM-generated text against a source and automatically correct identified errors.
Not ideal if your primary concern is grammar, style, or other linguistic qualities not directly related to factual consistency with a reference.
Stars
25
Forks
4
Language
Python
License
Apache-2.0
Category
Last pushed
May 23, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/intuit-ai-research/DCR-consistency"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
MMMU-Benchmark/MMMU
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal...
pat-jj/DeepRetrieval
[COLM’25] DeepRetrieval — 🔥 Training Search Agent by RLVR with Retrieval Outcome
lupantech/MathVista
MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts
x66ccff/liveideabench
[𝐍𝐚𝐭𝐮𝐫𝐞 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬] 🤖💡 LiveIdeaBench: Evaluating LLMs' Scientific Creativity and Idea...
ise-uiuc/magicoder
[ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct