ZuchniakK/MTKD
Multi-Teacher Knowledge Distillation, code for my PhD dissertation. I used knowledge distillation as a decision-fusion and compressing mechanism for ensemble models.
This project helps machine learning engineers and researchers deploy powerful deep learning models into real-time applications, especially on devices with limited resources. It takes multiple trained deep learning models (teachers) and compresses their collective knowledge into a single, smaller model (student) that maintains high accuracy but is far more efficient. This allows complex AI solutions for tasks like automated corrosion detection or wildfire smoke detection to run effectively on edge devices.
No commits in the last 6 months.
Use this if you need to deploy accurate deep learning models on devices with limited computational power or storage, without significantly sacrificing performance.
Not ideal if your existing single model is already lightweight enough for your deployment environment or if you have ample computational resources for large model ensembles.
Stars
27
Forks
3
Language
Jupyter Notebook
License
MIT
Category
Last pushed
May 19, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/ZuchniakK/MTKD"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Guang000/Awesome-Dataset-Distillation
A curated list of awesome papers on dataset distillation and related applications.
dkozlov/awesome-knowledge-distillation
Awesome Knowledge Distillation
SforAiDl/KD_Lib
A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of...
SakurajimaMaiii/ProtoKD
[ICASSP 2023] Prototype Knowledge Distillation for Medical Segmentation with Missing Modality
HikariTJU/LD
Localization Distillation for Object Detection (CVPR 2022, TPAMI 2023)