shufangxun/LLaVA-MoD
[ICLR 2025] LLaVA-MoD: Making LLaVA Tiny via MoE-Knowledge Distillation
This project helps machine learning engineers and researchers create smaller, more efficient Multimodal Language Models (MLLMs) that can understand both images and text. It takes a large, powerful MLLM as input and distills its knowledge to produce a 'tiny' MLLM that performs exceptionally well with significantly fewer computational resources. This is ideal for those needing to deploy advanced vision-language AI in resource-constrained environments.
223 stars. No commits in the last 6 months.
Use this if you need to build powerful AI models that can interpret both images and text, but require them to be compact and run efficiently on limited hardware.
Not ideal if you primarily work with text-only or image-only AI models, or if computational resources are not a significant constraint for your deployments.
Stars
223
Forks
16
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 31, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/shufangxun/LLaVA-MoD"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
scaleapi/llm-engine
Scale LLM Engine public repository
AGI-Arena/MARS
The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models
modelscope/easydistill
a toolkit on knowledge distillation for large language models
AGI-Edgerunners/LLM-Adapters
Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient...
Wang-ML-Lab/bayesian-peft
Bayesian Low-Rank Adaptation of LLMs: BLoB [NeurIPS 2024] and TFB [NeurIPS 2025]