Nota-NetsPresso/shortened-llm
Compressed LLMs for Efficient Text Generation [ICLR'24 Workshop]
This project helps machine learning engineers and researchers reduce the size and improve the speed of Large Language Models (LLMs) like LLaMA and Vicuna. By strategically removing parts of the model (depth pruning), it takes an existing LLM and outputs a smaller, faster version that still performs well. This is useful for anyone working with LLMs who needs to deploy them efficiently on limited hardware or for faster processing.
No commits in the last 6 months.
Use this if you need to make Large Language Models run faster and require less memory while maintaining strong performance for text generation and understanding tasks.
Not ideal if you are a casual user looking for an out-of-the-box, easy-to-use application and not a developer or researcher familiar with machine learning workflows and model deployment.
Stars
90
Forks
12
Language
Python
License
—
Category
Last pushed
Sep 13, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/Nota-NetsPresso/shortened-llm"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Higher-rated alternatives
ModelTC/LightCompress
[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs,...
p-e-w/heretic
Fully automatic censorship removal for language models
Orion-zhen/abliteration
Make abliterated models with transformers, easy and fast
YerbaPage/LongCodeZip
LongCodeZip: Compress Long Context for Code Language Models [ASE2025]
locuslab/wanda
A simple and effective LLM pruning approach.