guyyariv/vLMIG
This repo contains the official PyTorch implementation of vLMIG: Improving Visual Commonsense in Language Models via Multiple Image Generation
This project helps improve how large language models (LLMs) understand the world by integrating visual information. It takes your text prompts, generates multiple relevant images, and then blends these visual insights with the LLM's text-based reasoning. The result is an LLM that can better answer questions requiring both common sense and visual understanding. This is for researchers and developers working on advanced AI systems who want to enhance their models' reasoning capabilities.
No commits in the last 6 months.
Use this if you need to enhance an existing large language model's ability to perform visual commonsense reasoning, question answering, or reading comprehension by grounding it in visual information.
Not ideal if you are looking for a pre-trained, ready-to-use multimodal application without needing to integrate or fine-tune models yourself.
Stars
17
Forks
—
Language
Python
License
MIT
Category
Last pushed
Jul 01, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/guyyariv/vLMIG"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
open-mmlab/mmpretrain
OpenMMLab Pre-training Toolbox and Benchmark
facebookresearch/mmf
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
adambielski/siamese-triplet
Siamese and triplet networks with online pair/triplet mining in PyTorch
HuaizhengZhang/Awsome-Deep-Learning-for-Video-Analysis
Papers, code and datasets about deep learning and multi-modal learning for video analysis
KaiyangZhou/pytorch-vsumm-reinforce
Unsupervised video summarization with deep reinforcement learning (AAAI'18)