actypedef/ARCQuant
Code for the paper "ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs"
This tool helps AI developers and researchers improve the accuracy of large language models (LLMs) when using highly efficient, low-precision number formats like NVFP4. It takes your existing LLM and configuration, and outputs a quantized LLM that maintains high accuracy while enabling faster and more memory-efficient inference. It's designed for machine learning engineers and researchers working on deploying LLMs in resource-constrained environments.
Use this if you need to run large language models more efficiently on hardware, but are struggling to maintain model accuracy when using low-precision quantization methods like NVFP4.
Not ideal if you are not working with large language models, or if you do not require specialized low-bit quantization for performance optimization.
Stars
18
Forks
3
Language
Cuda
License
—
Category
Last pushed
Mar 03, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/actypedef/ARCQuant"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
bitsandbytes-foundation/bitsandbytes
Accessible large language models via k-bit quantization for PyTorch.
intel/neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model...
dropbox/hqq
Official implementation of Half-Quadratic Quantization (HQQ)
OpenGVLab/OmniQuant
[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
Hsu1023/DuQuant
[NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger...