lilakk/BooookScore
A package to generate summaries of long-form text and evaluate the coherence of these summaries. Official package for our ICLR 2024 paper, "BooookScore: A systematic exploration of book-length summarization in the era of LLMs".
This project helps researchers and practitioners evaluate the quality of AI-generated summaries for very long texts, like entire books. You input full book texts and the AI-generated summaries, and it outputs a coherence score for each summary. This is ideal for academics and researchers studying large language models and their summarization capabilities.
130 stars. No commits in the last 6 months. Available on PyPI.
Use this if you need to systematically generate and assess the coherence of summaries for extremely long documents using various large language models.
Not ideal if you're looking for a simple tool to generate short summaries of typical articles or documents without needing detailed quality evaluation.
Stars
130
Forks
10
Language
Python
License
MIT
Category
Last pushed
Oct 01, 2024
Commits (30d)
0
Dependencies
11
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/lilakk/BooookScore"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
openfactcheck-research/openfactcheck
An Open-source Factuality Evaluation Demo for LLMs
several27/FakeNewsCorpus
A dataset of millions of news articles scraped from a curated list of data sources.
Cartus/Automated-Fact-Checking-Resources
Links to conference/journal publications in automated fact-checking (resources for the...
armingh2000/FactScoreLite
FactScoreLite is an implementation of the FactScore metric, designed for detailed accuracy...
manideep2510/siamese-BERT-fake-news-detection-LIAR
Triple Branch BERT Siamese Network for fake news classification on LIAR-PLUS dataset in PyTorch