Mattdl/ContinualEvaluation
[Spotlight ICLR 2023 paper] Continual evaluation for lifelong learning with neural networks, identifying the stability gap.
When developing or deploying AI models that learn continuously over time, it's crucial to understand how well they retain old knowledge while learning new tasks. This tool helps machine learning engineers and researchers assess the real-time performance of their lifelong learning models, providing fine-grained insights into how accuracy changes at every learning step. It takes in your model and training data, and outputs detailed performance metrics that highlight potential drops in stability.
No commits in the last 6 months.
Use this if you are developing AI systems that learn continually and need a more rigorous way to evaluate their performance stability, especially for safety-critical applications.
Not ideal if you are working with traditional machine learning models that are trained once and then deployed without continuous updates.
Stars
35
Forks
4
Language
Python
License
MIT
Category
Last pushed
Apr 02, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/Mattdl/ContinualEvaluation"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
aimagelab/mammoth
An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of...
LAMDA-CL/PyCIL
PyCIL: A Python Toolbox for Class-Incremental Learning
GMvandeVen/continual-learning
PyTorch implementation of various methods for continual learning (XdG, EWC, SI, LwF, FROMP, DGR,...
LAMDA-CL/LAMDA-PILOT
🎉 PILOT: A Pre-trained Model-Based Continual Learning Toolbox
mmasana/FACIL
Framework for Analysis of Class-Incremental Learning with 12 state-of-the-art methods and 3 baselines.