HKUDS/SepLLM
[ICML 2025] "SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator"
This project helps data scientists and AI/ML engineers make large language models (LLMs) run much faster and use less memory, especially for long texts. By identifying and compressing less important parts of the text into 'separator' tokens, it reduces the computational load. This means you can process longer documents or complex queries more efficiently, making LLMs more practical for real-world applications.
567 stars. No commits in the last 6 months.
Use this if you are a developer or researcher working with large language models and need to improve their inference speed, reduce memory usage (KV cache), or handle extremely long text sequences in applications like advanced chatbots or document analysis.
Not ideal if you are an end-user simply looking to use an existing LLM or if your primary concern is fine-tuning small models where efficiency is not a critical bottleneck.
Stars
567
Forks
46
Language
Python
License
—
Category
Last pushed
Jul 29, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/HKUDS/SepLLM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ZHZisZZ/dllm
dLLM: Simple Diffusion Language Modeling
pengzhangzhi/Open-dLLM
Open diffusion language model for code generation — releasing pretraining, evaluation,...
EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. ACM...
THUDM/LongWriter
[ICLR 2025] LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs
AIoT-MLSys-Lab/SVD-LLM
[ICLR 2025🔥] SVD-LLM & [NAACL 2025🔥] SVD-LLM V2