WisconsinAIVision/ViP-LLaVA
[CVPR2024] ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts
This tool helps researchers and developers make large multimodal models (LMMs) understand specific regions or objects within an image. You provide an image and visually highlight a region (a 'visual prompt'), and the model outputs a detailed text description or answers questions about that specific area. It's designed for those working on computer vision, AI research, and multimodal AI applications.
336 stars. No commits in the last 6 months.
Use this if you need a way to precisely tell an AI model which part of an image to focus on when asking questions or generating descriptions.
Not ideal if you're looking for an off-the-shelf application for end-users, as this project is a research framework for building and evaluating LMMs.
Stars
336
Forks
21
Language
Python
License
Apache-2.0
Category
Last pushed
Jul 17, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/WisconsinAIVision/ViP-LLaVA"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Higher-rated alternatives
TinyLLaVA/TinyLLaVA_Factory
A Framework of Small-scale Large Multimodal Models
zjunlp/EasyInstruct
[ACL 2024] An Easy-to-use Instruction Processing Framework for LLMs.
rese1f/MovieChat
[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
haotian-liu/LLaVA
[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
NVlabs/Eagle
Eagle: Frontier Vision-Language Models with Data-Centric Strategies