snu-mllab/GuidedQuant
Official PyTorch implementation of "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance" (ICML 2025)
This project helps machine learning engineers and researchers make large language models (LLMs) more efficient. By applying advanced quantization techniques, it takes a full-sized LLM and outputs a significantly smaller, faster model that still performs well. The key benefit is running powerful LLMs on less powerful hardware or with faster inference speeds.
No commits in the last 6 months.
Use this if you need to deploy large language models more efficiently, reducing memory footprint and speeding up inference without a significant drop in performance.
Not ideal if you are working with smaller models that don't face significant computational or memory constraints.
Stars
50
Forks
2
Language
Python
License
MIT
Category
Last pushed
Jul 06, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/snu-mllab/GuidedQuant"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
bitsandbytes-foundation/bitsandbytes
Accessible large language models via k-bit quantization for PyTorch.
intel/neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model...
dropbox/hqq
Official implementation of Half-Quadratic Quantization (HQQ)
OpenGVLab/OmniQuant
[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
Hsu1023/DuQuant
[NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger...