elephantmipt/bert-distillation
Distillation of BERT model with catalyst framework
This project helps machine learning engineers or data scientists compress large BERT-based language models. It takes an existing, well-trained BERT model and training data as input, and outputs a smaller, faster version of that model. This is useful for deploying language models on resource-constrained devices like mobile phones or for speeding up inference in applications.
No commits in the last 6 months.
Use this if you need to reduce the size and increase the inference speed of a BERT-based language model while maintaining most of its performance.
Not ideal if you are a non-technical user or if you require a simple, out-of-the-box solution without custom code.
Stars
78
Forks
7
Language
Python
License
MIT
Category
Last pushed
Jun 12, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/elephantmipt/bert-distillation"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
airaria/TextBrewer
A PyTorch-based knowledge distillation toolkit for natural language processing
sunyilgdx/NSP-BERT
The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original...
princeton-nlp/CoFiPruning
[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
kssteven418/LTP
[KDD'22] Learned Token Pruning for Transformers
georgian-io/Transformers-Domain-Adaptation
:no_entry: [DEPRECATED] Adapt Transformer-based language models to new text domains