Fairness Bias Auditing ML Frameworks

Tools and frameworks for measuring, auditing, and mitigating bias and fairness issues in ML models and datasets. Includes bias detection metrics, fairness evaluation pipelines, and debiasing algorithms. Does NOT include general ML ethics resources, responsible AI governance frameworks, or fairness theory papers without implementation code.

There are 60 fairness bias auditing frameworks tracked. 1 score above 70 (verified tier). The highest-rated is fairlearn/fairlearn at 78/100 with 2,213 stars. 2 of the top 10 are actively maintained.

Get all 60 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=fairness-bias-auditing&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	fairlearn/fairlearn A Python package to assess and improve fairness of machine learning models.	78	Verified	2,213	Python
2	Trusted-AI/AIF360 A comprehensive set of fairness metrics for datasets and machine learning...	69	Established	2,763	Python
3	microsoft/responsible-ai-toolbox Responsible AI Toolbox is a suite of tools providing model and data...	64	Established	1,737	TypeScript
4	holistic-ai/holisticai This is an open-source tool to assess and improve the trustworthiness of AI systems.	64	Established	104	Jupyter Notebook
5	EFS-OpenSource/Thetis Service to examine data processing pipelines (e.g., machine learning or deep...	49	Emerging	6	Python
6	IBM/inFairness PyTorch package to train and audit ML models for Individual Fairness	49	Emerging	66	Python
7	datamllab/awesome-fairness-in-ai A curated list of awesome Fairness in AI resources	49	Emerging	332	—
8	linkedin/LiFT The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables...	47	Emerging	173	Scala
9	DataResponsibly/Virny An in-depth performance profiling library for machine learning models	44	Emerging	17	Python
10	ResponsiblyAI/responsibly Toolkit for Auditing and Mitigating Bias and Fairness of Machine Learning Systems 🔎🤖🧰	44	Emerging	100	Python
11	debiai/DebiAI Bias detection and contextual evaluation tool for your AI projects	42	Emerging	30	Vue
12	cylynx/verifyml Open-source toolkit to help companies implement responsible AI workflows.	42	Emerging	23	Python
13	yongkaiwu/FairAI This is a collection of papers and other resources related to fairness.	41	Emerging	95	HTML
14	Libr-AI/fairlib A framework for assessing and improving classification fairness.	40	Emerging	33	Jupyter Notebook
15	pliang279/LM_bias [ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models	39	Emerging	61	Python
16	Ali-Alameer/AI_fairness This GitHub repository offers resources to create fair and unbiased AI...	39	Emerging	14	Jupyter Notebook
17	xuetf/KDD_CUP_2020_Debiasing_Rush Solution to the Debiasing Track of KDD CUP 2020	39	Emerging	160	Jupyter Notebook
18	adhit-r/fairmind Ethical AI Governance Platform \| Bias Detection \| Compliance \| Fairness...	38	Emerging	7	Python
19	EqualityAI/EqualityML Evidence-based tools and community collaboration to end algorithmic bias,...	38	Emerging	35	Jupyter Notebook
20	biaslyze-dev/biaslyze The NLP Bias Identification Toolkit	38	Emerging	39	Jupyter Notebook
21	microsoft/responsible-ai-toolbox-mitigations Python library for implementing Responsible AI mitigations.	37	Emerging	69	Jupyter Notebook
22	mirianfsilva/ai-fairness Notes, references and materials on AI Fairness that I found useful and...	37	Emerging	9	—
23	JMitnik/FacialDebiasing Debiasing: Mitigating Algorithmic Facial Bias repository	36	Emerging	7	Jupyter Notebook
24	guldoganozgur/ei_fairness Code for the paper "Equal Improvability: A New Fairness Notion Considering...	35	Emerging	7	Jupyter Notebook
25	brandeis-machine-learning/awesome-ml-fairness Papers and online resources related to machine learning fairness	35	Emerging	75	—
26	ys-zong/MEDFAIR [ICLR 2023 spotlight] MEDFAIR: Benchmarking Fairness for Medical Imaging	35	Emerging	73	Python
27	nick-jhlee/fair-manifold-pca Fast and Efficient MMD-based Fair PCA via Optimization over Stiefel Manifold...	35	Emerging	11	C++
28	radames/Machine-Learning-Ethics-References List of references about Machine Learning bias and ethics	33	Emerging	64	—
29	uiuctml/fair-classification Post-processing for fair classification	33	Emerging	16	Jupyter Notebook
30	aman179102/trust-aware A trust-aware, human-in-the-loop AI decision system that knows when not to...	32	Emerging	9	Python
31	DetoxAI/detoxai Python toolkit for debiasing neural networks in image classification tasks	31	Emerging	81	Python
32	romanlutz/ResponsibleAI A collection of news articles, books, and papers on Responsible AI cases....	31	Emerging	70	—
33	aimms/bias-in-ai Shows how machine learning bias can be illustrated by creating an AIMMS...	30	Emerging	1	Python
34	yuji-roh/fairbatch FairBatch: Batch Selection for Model Fairness (ICLR 2021)	29	Experimental	19	Python
35	nmaudet/fairdiv-indivisible-items A collection of simple algorithms for fair division of indivisible items	28	Experimental	3	Jupyter Notebook
36	Trustworthy-ML-Lab/Training_Trustworthy_LRM_with_Refine A new training framework for Trustworthy Large Reasoning Models	28	Experimental	4	Python
37	aida-ugent/fairret A fairness library in PyTorch.	27	Experimental	32	Python
38	Doleus/doleus Build confidence in your AI with systematic slice-based testing	27	Experimental	11	Python
39	MaxPaez/fairness-analysis-pima 📊 Analyze fairness in Machine Learning models using the Pima Diabetes...	22	Experimental	—	Python
40	Jatin-sehgal7060/Cognitivelens-AI-Human-Comparison 🤖 Analyze and compare human and AI decision-making to enhance trust,...	22	Experimental	—	Python
41	FreakyF/Neural-Injustice Experimental research platform for biometric correlation and algorithmic bias.	21	Experimental	—	TypeScript
42	SayMyName-BiasNaming/samyna-tmlr Source code supporting the paper "Say My Name: a Model's Bias Discovery Framework"	20	Experimental	1	—
43	jmdvinodjmd/SSB_MLHC Official code for the paper “Sample Selection Bias in Machine Learning for...	20	Experimental	2	Jupyter Notebook
44	Aqib121201/WohnFair-cryptographically-auditable-housing-allocation-system α-fair, cryptographically auditable housing allocation system — reproducible...	19	Experimental	2	Go
45	ArpitaRandive/Fairness-Aware-Reinforcement-Learning-Under-Distribution-Shift This project simulates a public resource allocation system inspired by...	19	Experimental	—	Python
46	Tonyhrule/Queer-Bias-LLMs This project implements a collaborative agent pipeline to detect and reduce...	19	Experimental	3	Python
47	messihjlee/RM-IAT Replication Materials for "Implicit Bias-Like Patterns in Reasoning Models"	19	Experimental	1	Jupyter Notebook
48	ValentinLafargue/FairnessDetails Fairness is in the details : an auditing pipeline for face datasets	17	Experimental	—	—
49	djvill/SLAC-Fairness Tools to assess fairness and mitigate unfairness in sociolinguistic auto-coding	15	Experimental	—	HTML
50	Shaw1011/AI-Ethics-Auditor Production-grade platform for auditing AI/ML models for fairness,...	14	Experimental	—	Python
51	DIDSR/bias.myti.report A visualization tool to facilitate the comparison of user-implemented bias...	14	Experimental	1	Jupyter Notebook
52	zroe1/toy-models-of-superposition A replication of "Toy Models of Superposition," a groundbreaking machine...	13	Experimental	9	Jupyter Notebook
53	emanzahrashah/hiring-bias-fairness-analysis An end-to-end data science project analyzing bias and fairness in an...	13	Experimental	—	Jupyter Notebook
54	citp/fairness_diagnostics Tool to visualize and diagnose fairness issues in machine learning	12	Experimental	5	JavaScript
55	autumninthecloud/IM_w_fairness Our group capstone project site for UC Berkeley's MIDS W210/Capstone.	12	Experimental	6	JavaScript
56	Team-Obisidan/Biases-im-Machine-Learning This German scientific poster (developed as HTML5/React web app) shows the...	12	Experimental	1	JavaScript
57	iurada/fairness_crossdomain Official repository of our work "Fairness meets Cross-Domain Learning: a...	12	Experimental	6	Python
58	AI-for-Business/FairDice Through an engaging classroom experiment featuring a simple game, we aim to...	11	Experimental	—	—
59	AlexKubiesa/area-under-the-margin-ranking Implementation of the research paper Identifying Mislabeled Data using the...	11	Experimental	—	Jupyter Notebook
60	lucianosb/sinteticoxl-bias tool to evaluate biases on the SinteticoXL Models	11	Experimental	—	Python

Comparisons in this category

fairlearn and AIF360 (78 vs 69) fairlearn and inFairness (78 vs 49) fairlearn and fairlib (78 vs 40) fairlearn and AI_fairness (78 vs 39) fairlearn and fairmind (78 vs 38) responsible-ai-toolbox and responsibly (64 vs 44) awesome-fairness-in-ai and AI_fairness (49 vs 39)