mu-cai/matryoshka-mm
Matryoshka Multimodal Models
This project helps people who work with visual data and text to get more efficient and accurate results from their AI models. It takes in images and text, then processes them together to produce a better understanding of the visual content, often for tasks like image description or visual question answering. Researchers, AI developers, and data scientists building multimodal AI applications would find this useful.
122 stars. No commits in the last 6 months.
Use this if you are developing or evaluating advanced AI models that need to understand both images and text simultaneously, and you need highly granular visual insights.
Not ideal if you are looking for a simple, off-the-shelf image recognition tool without multimodal capabilities or if your tasks are purely text-based.
Stars
122
Forks
9
Language
Python
License
Apache-2.0
Category
Last pushed
Jan 22, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/mu-cai/matryoshka-mm"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
TinyLLaVA/TinyLLaVA_Factory
A Framework of Small-scale Large Multimodal Models
zjunlp/EasyInstruct
[ACL 2024] An Easy-to-use Instruction Processing Framework for LLMs.
rese1f/MovieChat
[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
haotian-liu/LLaVA
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
NVlabs/Eagle
Eagle: Frontier Vision-Language Models with Data-Centric Strategies