xv44586/Knowledge-Distillation-NLP
some demos of Knowledge Distillation in NLP
This project helps machine learning engineers and NLP practitioners make their large language models run faster and use fewer computing resources. It takes an existing, high-performing large model (the 'teacher') and distills its knowledge into a smaller, more efficient model (the 'student'). The output is a smaller model that performs almost as well as the original but is much quicker and cheaper to deploy in real-world applications.
No commits in the last 6 months.
Use this if you have a large, accurate natural language processing (NLP) model that is too slow or resource-intensive for your production environment and you need to optimize it for deployment.
Not ideal if you are looking to train a new NLP model from scratch or if your primary concern is improving model accuracy rather than efficiency.
Stars
23
Forks
6
Language
Jupyter Notebook
License
—
Category
Last pushed
Dec 31, 2020
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/xv44586/Knowledge-Distillation-NLP"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
airaria/TextBrewer
A PyTorch-based knowledge distillation toolkit for natural language processing
sunyilgdx/NSP-BERT
The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original...
kssteven418/LTP
[KDD'22] Learned Token Pruning for Transformers
princeton-nlp/CoFiPruning
[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
georgian-io/Transformers-Domain-Adaptation
:no_entry: [DEPRECATED] Adapt Transformer-based language models to new text domains