isadrtdinov/understanding-large-lrs
Source code for NeurIPS-2024 paper "Where Do Large Learning Rates Lead Us"
This project helps machine learning researchers and practitioners understand how different initial learning rates affect the training and final performance of neural networks. It takes your neural network training configurations and outputs insights into the quality of the learned model, helping you achieve optimal generalization. This is for machine learning researchers, deep learning engineers, and data scientists who are fine-tuning neural network models.
No commits in the last 6 months.
Use this if you are trying to optimize the generalization performance of your neural networks and need to understand the impact of initial learning rates on model quality and feature learning.
Not ideal if you are looking for a plug-and-play solution for automatic learning rate tuning without delving into the underlying theoretical implications or local minima geometry.
Stars
7
Forks
—
Language
Jupyter Notebook
License
Apache-2.0
Category
Last pushed
Dec 14, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/isadrtdinov/understanding-large-lrs"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
kaanaksit/odak
Scientific computing library for optics, computer graphics and visual perception.
NVIDIA/torch-harmonics
Differentiable signal processing on the sphere for PyTorch
PreFab-Photonics/PreFab
Artificial nanofabrication of integrated photonic circuits using deep learning
MatthewFilipovich/torchoptics
Differentiable wave optics simulation library built on PyTorch
artificial-scientist-lab/XLuminA
XLuminA, a highly-efficient, auto-differentiating discovery framework for super-resolution microscopy.