andrescorrada/IntroductionToAlgebraicEvaluation
A collection of essays and code on algebraic methods to evaluate noisy judges on unlabeled test data.
This project offers a method to evaluate how well different decision-makers (like AI models or human experts) perform, even when you don't know the correct answers to the questions they're responding to. You input the observed agreements and disagreements between these 'noisy judges' on a test, and it helps you infer their individual correctness statistics. It's designed for anyone who needs to assess the reliability of multiple decision-makers on unlabeled data.
Use this if you need to reliably evaluate the performance of multiple human or machine agents, especially AI systems, on a test where the true answers are unknown.
Not ideal if you already have perfectly labeled data and are using traditional, supervised evaluation methods.
Stars
37
Forks
4
Language
Jupyter Notebook
License
CC0-1.0
Category
Last pushed
Feb 25, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/andrescorrada/IntroductionToAlgebraicEvaluation"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
stabgan/Multiple-Linear-Regression
Implementation of Multiple Linear Regression both in Python and R
SENATOROVAI/Normal-equation-solver-multiple-linear-regression-course
Multiple Linear Regression (MLR) models the linear relationship between a continuous dependent...
SENATOROVAI/Normal-equations-scalar-form-solver-simple-linear-regression-course
The normal equations for simple linear regression are a system of two linear equations used to...
SENATOROVAI/underfitting-overfitting-polynomial-regression-course
Underfitting and overfitting are critical concepts in machine learning, particularly when using...
MathWorks-Teaching-Resources/Machine-Learning-for-Regression
Interactive courseware module that introduces typical workflow, setup, and considerations...