garyfanhku/Galore-pytorch
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
This project helps machine learning engineers and researchers train very large language models more efficiently. It takes a large language model and training data as input, and produces a trained model while using significantly less memory. This allows for training models that would otherwise be too large for available hardware.
No commits in the last 6 months.
Use this if you are a machine learning practitioner struggling with out-of-memory errors when training large language models due to hardware limitations.
Not ideal if you are looking for a fully polished, production-ready library with comprehensive logging and real-world data training examples.
Stars
22
Forks
1
Language
Python
License
—
Category
Last pushed
Mar 07, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/garyfanhku/Galore-pytorch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
OptimalScale/LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
adithya-s-k/AI-Engineering.academy
Mastering Applied AI, One Concept at a Time
jax-ml/jax-llm-examples
Minimal yet performant LLM examples in pure JAX
young-geng/scalax
A simple library for scaling up JAX programs
riyanshibohra/TuneKit
Upload your data → Get a fine-tuned SLM. Free.