OSU-NLP-Group/AttrScore
Code, datasets, models for the paper "Automatic Evaluation of Attribution by Large Language Models"
This project helps evaluate how well a large language model's (LLM) answer is supported by a given source text. You provide an LLM's claim (query + answer) and a reference document, and it tells you if the claim is Attributable, Extrapolatory, or Contradictory. Anyone working with LLMs who needs to verify the factual accuracy of their outputs against source material would find this useful.
No commits in the last 6 months.
Use this if you need to automatically and systematically assess the factual basis and trustworthiness of information generated by large language models.
Not ideal if you are looking for a tool to generate text or improve the fluency of an LLM's output, as this focuses on evaluation, not generation.
Stars
56
Forks
2
Language
Python
License
MIT
Category
Last pushed
Jul 03, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/OSU-NLP-Group/AttrScore"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
filipnaudot/llmSHAP
llmSHAP: a multi-threaded explainability framework using Shapley values for LLM-based outputs.
microsoft/automated-brain-explanations
Generating and validating natural-language explanations for the brain.
CAS-SIAT-XinHai/CPsyCoun
[ACL 2024] CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework...
wesg52/universal-neurons
Universal Neurons in GPT2 Language Models
ICTMCG/LLM-for-misinformation-research
Paper list of misinformation research using (multi-modal) large language models, i.e., (M)LLMs.