INK-USC/sparse-distillation
Code for "Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models"
This project helps data scientists and machine learning engineers speed up text classification tasks. It takes large, pre-trained language models and unlabeled text data to produce a smaller, faster model that performs text classification with high accuracy. This is ideal for teams needing to deploy text classifiers efficiently without sacrificing performance.
No commits in the last 6 months.
Use this if you need to classify text data quickly and accurately, and have access to both labeled examples and a large corpus of unlabeled text.
Not ideal if you don't have a pre-trained RoBERTa model or a substantial amount of unlabeled text data to leverage for the distillation process.
Stars
12
Forks
3
Language
Python
License
MIT
Category
Last pushed
May 11, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/INK-USC/sparse-distillation"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
airaria/TextBrewer
A PyTorch-based knowledge distillation toolkit for natural language processing
sunyilgdx/NSP-BERT
The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original...
kssteven418/LTP
[KDD'22] Learned Token Pruning for Transformers
princeton-nlp/CoFiPruning
[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
georgian-io/Transformers-Domain-Adaptation
:no_entry: [DEPRECATED] Adapt Transformer-based language models to new text domains