YJiangcm/Lion
[EMNLP 2023] Lion: Adversarial Distillation of Proprietary Large Language Models
This project helps machine learning engineers and researchers create smaller, more efficient large language models (LLMs) that closely mimic the performance of powerful, proprietary LLMs like ChatGPT. It takes an existing instruction-following dataset and a proprietary teacher model, then produces a distilled, compact LLM. This process is ideal for those who need to deploy performant LLMs with fewer computational resources or under stricter privacy constraints.
212 stars. No commits in the last 6 months.
Use this if you need to build a smaller, faster language model that can perform complex instruction-following tasks almost as well as a large proprietary model, but with reduced operational costs.
Not ideal if you're looking for a ready-to-use application or a no-code solution, as this project requires significant technical expertise in machine learning and GPU infrastructure to implement.
Stars
212
Forks
19
Language
Python
License
MIT
Category
Last pushed
Feb 11, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/YJiangcm/Lion"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scaleapi/llm-engine
Scale LLM Engine public repository
AGI-Arena/MARS
The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models
modelscope/easydistill
a toolkit on knowledge distillation for large language models
AGI-Edgerunners/LLM-Adapters
Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient...
Wang-ML-Lab/bayesian-peft
Bayesian Low-Rank Adaptation of LLMs: BLoB [NeurIPS 2024] and TFB [NeurIPS 2025]