feyzaakyurek/bbnli
Bias Benchmark for Natural Language Inference. Code repo for the Findings of NAACL 2022 paper "On Measuring Social Biases in Prompt-Based Multi-Task Learning".
This tool helps researchers and developers evaluate large language models for social biases related to gender, race, and religion. You input a set of premises and hypotheses, and it outputs bias scores indicating how much the model's inferences reflect societal biases. It is designed for those who work with or develop AI language models and need to ensure fairness.
No commits in the last 6 months.
Use this if you are developing or fine-tuning large language models and need to systematically measure and understand their inherent social biases.
Not ideal if you are looking for a general-purpose natural language inference tool or a solution for detecting bias in human-generated text.
Stars
15
Forks
1
Language
Python
License
MIT
Category
Last pushed
Apr 28, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/feyzaakyurek/bbnli"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
dccuchile/wefe
WEFE: The Word Embeddings Fairness Evaluation Framework. WEFE is a framework that standardizes...
dreji18/Fairness-in-AI
Detecting Bias and ensuring Fairness in AI solutions
amazon-science/bold
Dataset associated with "BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language...
dhfbk/variationist
Variationist: Exploring Multifaceted Variation and Bias in Written Language Data (ACL 2024 demo track)
soarsmu/BiasFinder
BiasFinder | IEEE TSE | Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systems