ironjr/grokfast
Official repository for the paper "Grokfast: Accelerated Grokking by Amplifying Slow Gradients"
This tool helps machine learning researchers and practitioners accelerate the "grokking" phenomenon in their models. When training deep learning models, sometimes they overfit to the training data perfectly before suddenly generalizing well to new, unseen data, which is called grokking. This tool takes your existing PyTorch model's gradients and processes them to drastically speed up this generalization process, potentially reducing training time significantly.
578 stars. No commits in the last 6 months.
Use this if your deep learning model exhibits grokking, where it overfits the training data for a long time before finally generalizing, and you want to speed up that generalization phase.
Not ideal if your model does not exhibit the grokking phenomenon or if you are not working with PyTorch models.
Stars
578
Forks
50
Language
Python
License
MIT
Category
Last pushed
Jun 28, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/ironjr/grokfast"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
PaddlePaddle/Paddle
PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice...
fastai/fastai
The fastai deep learning library
openvinotoolkit/openvino_notebooks
📚 Jupyter notebook tutorials for OpenVINO™
PaddlePaddle/docs
Documentations for PaddlePaddle
msuzen/bristol
Parallel random matrix tools and complexity for deep learning