qiangsiwei/bert_distill
BERT distillation（基于BERT的蒸馏实验 ）
This project helps machine learning practitioners create smaller, faster text classification models without sacrificing too much accuracy. It takes a large, pre-trained BERT model and a dataset (like customer reviews), then transfers BERT's knowledge to a more lightweight model such as TextCNN or BiLSTM. The output is a smaller model that can classify text with good performance, suitable for deployment in environments with limited resources.
314 stars. No commits in the last 6 months.
Use this if you need to deploy text classification capabilities on devices or systems with computational constraints, where a full BERT model would be too slow or resource-intensive.
Not ideal if your primary goal is to achieve the absolute highest possible accuracy, or if you have ample computational resources and inference speed is not a critical concern.
Stars
314
Forks
82
Language
Python
License
—
Category
Last pushed
Jul 30, 2020
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/qiangsiwei/bert_distill"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
airaria/TextBrewer
A PyTorch-based knowledge distillation toolkit for natural language processing
sunyilgdx/NSP-BERT
The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original...
kssteven418/LTP
[KDD'22] Learned Token Pruning for Transformers
princeton-nlp/CoFiPruning
[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
georgian-io/Transformers-Domain-Adaptation
:no_entry: [DEPRECATED] Adapt Transformer-based language models to new text domains