DragonLiu1995/video-to-audio-through-text
[NeurIPS 2024] Code, Dataset, Samples for the VATT paper “ Tell What You Hear From What You See - Video to Audio Generation Through Text”
This project helps video creators, educators, and content strategists automatically add realistic audio to silent videos or generate audio descriptions. It takes a video and an optional text prompt, then produces relevant audio and, if desired, a text description of that audio. This is perfect for anyone needing to enhance video content with rich, context-aware sound without manual sound design.
No commits in the last 6 months.
Use this if you need to generate high-quality, context-relevant audio or audio descriptions for videos, especially when you want to guide the audio generation with specific text prompts.
Not ideal if you're looking for a simple drag-and-drop tool for quick, casual video edits, as it requires some technical setup and a GPU.
Stars
36
Forks
2
Language
Python
License
—
Category
Last pushed
Jul 24, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/generative-ai/DragonLiu1995/video-to-audio-through-text"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Mrkomiljon/awesome-generative-ai
Multimodal generative AI resources : talking heads, STT, TTS, image & video generation, and more.
NVIDIA/Maya-ACE
Maya-ACE: A Reference Client Implementation for NVIDIA ACE Audio2Face Service
OpenVGLab/OmniLottie
[CVPR 2026🔥] 🧑🎨 OmniLottie, an open-sourced multi-modal instructed vector animation generator...
jdh-algo/JoyHallo
JoyHallo: Digital human model for Mandarin
michaelzhang-ai/Speech2Video
ACCV 2020 "Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses"