2toinf/IVM
[NeurIPS-2024] The offical Implementation of "Instruction-Guided Visual Masking"
This tool helps improve how AI models, like those used in robotics or for answering questions about images, understand and react to visual information. You provide an image and a specific instruction (e.g., "pick up the red cup"), and it generates a mask that highlights only the relevant parts of the image, allowing the AI to focus better. This is for AI developers and researchers building or enhancing multimodal AI systems, especially those integrating vision and language.
No commits in the last 6 months.
Use this if you are developing AI models that need to accurately interpret complex visual instructions and want to improve their ability to focus on specific objects or regions within an image.
Not ideal if you are a non-developer looking for a ready-to-use application for image editing or general object detection without integrating it into an AI model workflow.
Stars
42
Forks
2
Language
Jupyter Notebook
License
Apache-2.0
Category
Last pushed
Nov 15, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/2toinf/IVM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
KimMeen/Time-LLM
[ICLR 2024] Official implementation of " 🦙 Time-LLM: Time Series Forecasting by Reprogramming...
om-ai-lab/VLM-R1
Solve Visual Understanding with Reinforced VLMs
bytedance/SALMONN
SALMONN family: A suite of advanced multi-modal LLMs
NVlabs/OmniVinci
OmniVinci is an omni-modal LLM for joint understanding of vision, audio, and language.
fixie-ai/ultravox
A fast multimodal LLM for real-time voice