ML Robustness Frameworks

Tools and frameworks for testing, evaluating, and improving the robustness of ML models against corruptions, adversarial perturbations, domain shifts, and distribution changes. Does NOT include general model evaluation, fairness/bias mitigation, or privacy-preserving machine learning outside robustness contexts.

There are 35 ml robustness frameworks tracked. 4 score above 50 (established tier). The highest-rated is namkoong-lab/dro at 55/100 with 157 stars.

Get all 35 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=ml-robustness-frameworks&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	namkoong-lab/dro A package of distributionally robust optimization (DRO) methods. Implemented...	55	Established	157	Jupyter Notebook
2	MinghuiChen43/awesome-trustworthy-deep-learning A curated list of trustworthy deep learning papers. Daily updating...	52	Established	382	—
3	neu-autonomy/nfl_veripy Formal Verification of Neural Feedback Loops (NFLs)	52	Established	83	Python
4	THUDM/grb Graph Robustness Benchmark: A scalable, unified, modular, and reproducible...	52	Established	99	Python
5	ADA-research/VERONA A lightweight Python package for setting up robustness experiments and to...	49	Emerging	49	Python
6	hendrycks/robustness Corruption and Perturbation Robustness (ICLR 2019)	48	Emerging	1,139	Python
7	iutzeler/skwdro Distributionally robust machine learning with Pytorch and Scikit-learn wrappers	46	Emerging	19	Python
8	microsoft/robustdg Toolkit for building machine learning models that generalize to unseen...	44	Emerging	175	Python
9	alibaba/easyrobust EasyRobust: an Easy-to-use library for state-of-the-art Robust Computer...	42	Emerging	338	Jupyter Notebook
10	jiachens/ModelNet40-C Repo for "Benchmarking Robustness of 3D Point Cloud Recognition against...	40	Emerging	216	Python
11	Iyengar-Lab/E2E-DRO End-to-end distributionally robust optimization	40	Emerging	38	Python
12	RuntianZ/doro Distributional and Outlier Robust Optimization (ICML 2021)	37	Emerging	28	Python
13	MLI-lab/Robustness-CS Measuring the robustness of compressive sensing methods (including...	36	Emerging	32	Jupyter Notebook
14	RyanLucas3/HR_Neural_Networks Certified robustness of deep neural networks	36	Emerging	19	Python
15	ShoumikSaha/DRSM DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified...	36	Emerging	14	Python
16	CausalML/doubly-robust-dropel Off-Policy Evaluation and Learning that is both Doubly Robust and...	35	Emerging	10	Jupyter Notebook
17	val-iisc/GD-UAP Generalized Data-free Universal Adversarial Perturbations	35	Emerging	73	Python
18	sarthaxxxxx/AVROBUSTBENCH Benchmarking robustness of audio-visual recognition models at test-time	34	Emerging	10	Jupyter Notebook
19	jh-jeong/smoothmix Code for the paper "SmoothMix: Training Confidence-calibrated Smoothed...	33	Emerging	21	Roff
20	ByungKwanLee/Super-Fast-Adversarial-Training Official PyTorch Implementation Code for Developing Super Fast Adversarial...	32	Emerging	33	Python
21	pmichel31415/P-DRO Code for the papers "Modeling the Second Player in Distributionally Robust...	32	Emerging	29	Python
22	BBVA/mercury-robust mercury-robust is a framework to perform robust testing on ML models and...	31	Emerging	20	Jupyter Notebook
23	Trustworthy-ML-Lab/corrupting_neuron_explanations [ICCV 23] Evaluating robustness of neuron explanation methods	31	Emerging	4	Jupyter Notebook
24	AI-secure/VeriGauge A united toolbox for running major robustness verification approaches for...	27	Experimental	90	C
25	alejandrods/Analysis-of-the-robustness-of-NMF-algorithms Analysis of the robustness of non-negative matrix factorization (NMF)...	25	Experimental	11	Jupyter Notebook
26	SebChw/Actually-Robust-Training Actually Robust Training - Tool Inspired by Andrej Karpathy "Recipe for...	24	Experimental	43	Python
27	dedeswim/vits-robustness-torch Code for the paper "A Light Recipe to Train Robust Vision Transformers" [SaTML 2023]	23	Experimental	54	Jupyter Notebook
28	nmndeep/revisiting-at [NeurIPS 2023] Code for the paper "Revisiting Adversarial Training for...	23	Experimental	39	Python
29	katelyn98/CorruptionRobustness We investigated corruption robustness across different architectures...	23	Experimental	16	Jupyter Notebook
30	im-ethz/pub-gdu4dg Gated Domain Units (GDU) aim to make your deep learning models robust...	20	Experimental	7	Jupyter Notebook
31	or4k2l/robust-vision Production-ready framework for training robust computer vision models....	20	Experimental	1	Python
32	Yangyi-Chen/PaperList-Trustworthy-Applications Mostly recording papers about models' trustworthy applications. Intending to...	18	Experimental	21	—
33	Trustworthy-ML-Lab/Robust_HighUtil_Smoothed_DRL [ICML 24] S-DQN and S-PPO: Robust smoothed deep RL agents without...	14	Experimental	5	Python
34	MK-Wireless/coded-neural-networks Structured redundancy for neural networks using coding-theoretic principles.	13	Experimental	—	—
35	sghosh-04/cnn-generalization-dataset-shift independent research on CNN robustness under dataset shift — EfficientNet,...	11	Experimental	—	Jupyter Notebook

Comparisons in this category

dro and E2E-DRO (55 vs 40) dro and skwdro (55 vs 46)