lyuchenyang/Macaw-LLM
Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
This project helps AI researchers and developers working with large language models to integrate various types of unstructured data. It takes inputs like images, videos, audio clips, and text, then processes and aligns them to be understood by a language model. The output is a multi-modal language model capable of processing and generating responses based on diverse data types.
1,593 stars. No commits in the last 6 months.
Use this if you are developing advanced AI models and need to combine information from images, videos, audio, and text for a unified language understanding system.
Not ideal if you are looking for a ready-to-use application or API for end-user tasks, as this is a foundational model for further AI development.
Stars
1,593
Forks
132
Language
Python
License
Apache-2.0
Category
Last pushed
Jan 01, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/lyuchenyang/Macaw-LLM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
kyegomez/RT-X
Pytorch implementation of the models RT-1-X and RT-2-X from the paper: "Open X-Embodiment:...
kyegomez/PALI3
Implementation of PALI3 from the paper PALI-3 VISION LANGUAGE MODELS: SMALLER, FASTER, STRONGER"
chuanyangjin/MMToM-QA
[🏆Outstanding Paper Award at ACL 2024] MMToM-QA: Multimodal Theory of Mind Question Answering
Muennighoff/vilio
🥶Vilio: State-of-the-art VL models in PyTorch & PaddlePaddle
kyegomez/PALM-E
Implementation of "PaLM-E: An Embodied Multimodal Language Model"