aangelopoulos/conformal-risk
Conformal prediction for controlling monotonic risk functions. Simple accompanying PyTorch code for conformal risk control in computer vision and natural language processing.
This tool helps machine learning engineers and researchers build models that maintain a desired level of performance, especially when it's critical to control specific types of errors like false negatives or incorrect predictions. You input a trained machine learning model and a target error rate, and it outputs a calibrated model that reliably keeps the error rate below your set limit. It's designed for those developing or deploying AI systems in areas like medical imaging, multi-label classification, or question answering.
No commits in the last 6 months.
Use this if you need to guarantee that your machine learning model’s error rate (like false negatives, or distance from the true answer) stays below a specified threshold, rather than just optimizing for average performance.
Not ideal if you are looking for a general-purpose model training framework or if your primary goal is to improve overall model accuracy without specific risk constraints.
Stars
79
Forks
11
Language
Python
License
MIT
Category
Last pushed
Jan 23, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/aangelopoulos/conformal-risk"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
princeton-nlp/SimCSE
[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821
n-waves/multifit
The code to reproduce results from paper "MultiFiT: Efficient Multi-lingual Language Model...
yxuansu/SimCTG
[NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation
alibaba-edu/simple-effective-text-matching
Source code of the ACL2019 paper "Simple and Effective Text Matching with Richer Alignment Features".
Shark-NLP/OpenICL
OpenICL is an open-source framework to facilitate research, development, and prototyping of...