SCZwangxiao/RTQ-MM2023
ACM Multimedia 2023 (Oral) - RTQ: Rethinking Video-language Understanding Based on Image-text Model
This project helps researchers and developers working with video content to better understand and describe what's happening within those videos. It takes raw video files and text descriptions or questions as input, and outputs concise captions, answers to questions, or relevant video clips based on text queries. This is ideal for those involved in advanced AI research or building applications that analyze video content.
No commits in the last 6 months.
Use this if you are a researcher or AI developer working on understanding complex video content and want to achieve state-of-the-art performance in tasks like text-to-video retrieval, video captioning, or video question answering without extensive video-specific pre-training.
Not ideal if you need a simple, out-of-the-box solution for basic video editing or categorization, as this is a research framework requiring technical expertise to implement and adapt.
Stars
16
Forks
3
Language
Python
License
BSD-3-Clause
Category
Last pushed
Jan 31, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/SCZwangxiao/RTQ-MM2023"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
kyegomez/RT-X
Pytorch implementation of the models RT-1-X and RT-2-X from the paper: "Open X-Embodiment:...
kyegomez/PALI3
Implementation of PALI3 from the paper PALI-3 VISION LANGUAGE MODELS: SMALLER, FASTER, STRONGER"
chuanyangjin/MMToM-QA
[🏆Outstanding Paper Award at ACL 2024] MMToM-QA: Multimodal Theory of Mind Question Answering
lyuchenyang/Macaw-LLM
Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
Muennighoff/vilio
🥶Vilio: State-of-the-art VL models in PyTorch & PaddlePaddle