VITA-Group/Q-GaLore
Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
This project offers a memory-efficient way to train large language models, like LLaMA-7B, even on GPUs with limited memory (e.g., 16GB). It takes your model configuration and training data, then outputs a fully trained model that required significantly less memory during the training process. This is ideal for machine learning engineers and researchers working on large-scale AI models.
203 stars. No commits in the last 6 months.
Use this if you need to pre-train or fine-tune large language models but are constrained by the memory capacity of your GPUs.
Not ideal if you are working with small models or have access to ample high-memory GPU resources, as the setup might add unnecessary complexity.
Stars
203
Forks
19
Language
Python
License
Apache-2.0
Category
Last pushed
Jul 17, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/VITA-Group/Q-GaLore"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
bitsandbytes-foundation/bitsandbytes
Accessible large language models via k-bit quantization for PyTorch.
intel/neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model...
dropbox/hqq
Official implementation of Half-Quadratic Quantization (HQQ)
OpenGVLab/OmniQuant
[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
Hsu1023/DuQuant
[NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger...