griff4692/calibrating-summaries
This is the official PyTorch codebase for the ACL 2023 paper: "What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarization".
This toolkit helps scientific researchers and content creators improve the quality of automatically generated summaries for long-form scientific documents like chemistry papers or clinical studies. You provide the original scientific text and its summary, and the toolkit helps refine the summary to be more relevant or factually accurate. It's designed for anyone who relies on automated summarization for scientific content and needs to ensure high quality.
No commits in the last 6 months.
Use this if you need to rigorously evaluate and improve the relevance or faithfulness of machine-generated summaries for long scientific articles.
Not ideal if you are looking for a general-purpose summarization tool for non-scientific text or if you don't have existing summaries to calibrate against.
Stars
9
Forks
1
Language
Python
License
—
Category
Last pushed
Aug 14, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/griff4692/calibrating-summaries"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
princeton-nlp/SimCSE
[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
n-waves/multifit
The code to reproduce results from paper "MultiFiT: Efficient Multi-lingual Language Model...
yxuansu/SimCTG
[NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation
alibaba-edu/simple-effective-text-matching
Source code of the ACL2019 paper "Simple and Effective Text Matching with Richer Alignment Features".
Shark-NLP/OpenICL
OpenICL is an open-source framework to facilitate research, development, and prototyping of...