Model Confidence Calibration ML Frameworks

Tools and techniques for calibrating neural network confidence scores and probability predictions, including post-hoc methods, metrics, and frameworks. Does NOT include general uncertainty quantification, Bayesian methods, or confidence intervals unrelated to classifier calibration.

There are 21 model confidence calibration frameworks tracked. 3 score above 50 (established tier). The highest-rated is facebookincubator/MCGrad at 55/100 with 18 stars.

Get all 21 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=model-confidence-calibration&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	facebookincubator/MCGrad MCGrad is a scalable and easy-to-use tool for multicalibration. It ensures...	55	Established	18	Jupyter Notebook
2	dholzmueller/probmetrics Post-hoc calibration methods and metrics for classification	54	Established	53	Python
3	gpleiss/temperature_scaling A simple way to calibrate your neural network.	50	Established	1,167	Python
4	yfzhang114/Generalization-Causality 关于domain generalization，domain...	44	Emerging	1,238	—
5	hollance/reliability-diagrams Reliability diagrams visualize whether a classifier model needs calibration	41	Emerging	167	Jupyter Notebook
6	Affirm/splinator Splinator: probabilistic calibration with regression splines	41	Emerging	24	Python
7	DiTEC-project/DiTEC_WDN_dataset This repository contains parameter generation, simulation, and encapsulation...	39	Emerging	9	Python
8	uncbiag/LTS Local Temperature Scaling for Probability Calibration	38	Emerging	22	Python
9	by-liu/MbLS Code of our method MbLS (Margin-based Label Smoothing) for network...	37	Emerging	50	Python
10	lorenzofamiglini/CalFram Calibration Framework for Machine Learning and Deep Learning	37	Emerging	16	Python
11	VIDA-NYU/pycalibrate pycalibrate is a Python library to visually analyze model calibration in...	36	Emerging	17	Jupyter Notebook
12	mdca-loss/MDCA-Calibration [CVPR 2022] Official code for the paper: "A Stitch in Time Saves Nine: A...	36	Emerging	33	Python
13	divelab/LECI The implementation of "Joint Learning of Label and Environment Causal...	33	Emerging	22	Python
14	kirill-vish/Beyond-INet Code for experiments for "ConvNet vs Transformer, Supervised vs CLIP: Beyond...	32	Emerging	102	Python
15	WenjianHuang93/h-Calibration h-calibration: post-hoc calibration for deep learning classifier	28	Experimental	28	Python
16	aai-institute/kyle A library for calibrating classifiers and computing calibration metrics	27	Experimental	14	Jupyter Notebook
17	jsbaan/calibration-on-disagreement-data Code accompanying the EMNLP 2022 paper "Stop Measuring Calibration When...	24	Experimental	5	Jupyter Notebook
18	maheeppurohit/epistemic-weight-engine Epistemic Weight Engine (EWE) — A pre-update gating mechanism for...	22	Experimental	—	Python
19	martinferianc/noise Investigation of how noise perturbations impact neural network calibration...	20	Experimental	6	Shell
20	MohammadErfan-Jabbari/CNN-Calibration Investigating calibration in CNNs: Temperature scaling for reliable...	17	Experimental	—	HTML
21	hmcalister/Neural-Network-Weight-Importance-Measures An investigation into weight importance measures in neural networks,...	10	Experimental	2	Jupyter Notebook