andreped/GradientAccumulator
:dart: Gradient Accumulation for TensorFlow 2
When training large deep learning models with TensorFlow 2, you often hit GPU memory limits when using big batch sizes. This project lets you simulate much larger batch sizes than your GPU can natively handle. It takes your existing TensorFlow model and processes batches in smaller chunks, accumulating the gradients until a full virtual batch is complete. This helps deep learning researchers and practitioners train complex models that would otherwise be infeasible due to hardware constraints.
No commits in the last 6 months.
Use this if you are a deep learning researcher or practitioner using TensorFlow 2 and need to train models with very large batch sizes but are limited by your GPU's memory.
Not ideal if you are not using TensorFlow 2 or are not dealing with GPU memory limitations during model training.
Stars
53
Forks
11
Language
Python
License
MIT
Category
Last pushed
Feb 11, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/andreped/GradientAccumulator"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
mlverse/torch
R Interface to Torch
modern-fortran/neural-fortran
A parallel framework for deep learning
Beliavsky/Fortran-code-on-GitHub
Directory of Fortran codes on GitHub, arranged by topic
Cambridge-ICCS/FTorch
A library for directly calling PyTorch ML models from Fortran.
NVIDIA/TorchFort
An Online Deep Learning Interface for HPC programs on NVIDIA GPUs