golololologol/LLM-Distillery
A pipeline for LLM knowledge distillation
This tool helps developers make large language models (LLMs) smaller and more efficient without losing their core knowledge. You provide one or more larger, more capable 'teacher' LLMs and a dataset of instructions or text. The tool then produces a smaller 'student' LLM that has learned from the teachers, which is ideal for deployment in resource-constrained environments. This is for machine learning engineers and AI solution architects looking to optimize LLM performance and cost.
112 stars. No commits in the last 6 months.
Use this if you need to create a compact, efficient version of a larger language model for faster inference or reduced computational costs, leveraging knowledge from one or many existing models.
Not ideal if you are looking for a tool to train a large language model from scratch, as this focuses on distilling existing models into smaller ones.
Stars
112
Forks
14
Language
Python
License
Apache-2.0
Category
Last pushed
Apr 02, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/golololologol/LLM-Distillery"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Higher-rated alternatives
scaleapi/llm-engine
Scale LLM Engine public repository
AGI-Arena/MARS
The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models
modelscope/easydistill
a toolkit on knowledge distillation for large language models
AGI-Edgerunners/LLM-Adapters
Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient...
Wang-ML-Lab/bayesian-peft
Bayesian Low-Rank Adaptation of LLMs: BLoB [NeurIPS 2024] and TFB [NeurIPS 2025]