namkoong-lab/whyshift
A python package providing a benchmark with various specified distribution shift patterns.
Researchers and data scientists building machine learning models need to understand how well their models perform when the underlying data patterns shift. This tool provides a benchmark with real-world tabular datasets like income, public health insurance, mobility, taxi trip durations, and accident severity. It helps evaluate how robust a model is to various specified distribution shifts, giving insights into its reliability beyond the training environment. The primary users are researchers focused on fair machine learning, robust AI, and out-of-distribution generalization.
No commits in the last 6 months. Available on PyPI.
Use this if you are a researcher who needs to rigorously test your machine learning models against known, controlled distribution shifts on diverse real-world datasets to understand their generalization capabilities.
Not ideal if you are looking for a general-purpose data preprocessing library or a tool to automatically correct for arbitrary data shifts without defined patterns.
Stars
59
Forks
4
Language
Jupyter Notebook
License
—
Category
Last pushed
Nov 27, 2023
Commits (30d)
0
Dependencies
9
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/namkoong-lab/whyshift"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
adapt-python/adapt
Awesome Domain Adaptation Python Toolbox
corenel/pytorch-adda
A PyTorch implementation for Adversarial Discriminative Domain Adaptation
jindongwang/transferlearning
Transfer learning / domain adaptation / domain generalization / multi-task learning etc. Papers,...
thuml/Transfer-Learning-Library
Transfer Learning Library for Domain Adaptation, Task Adaptation, and Domain Generalization
KaiyangZhou/Dassl.pytorch
A PyTorch toolbox for domain generalization, domain adaptation and semi-supervised learning.