Lip Reading Synthesis ML Frameworks
Tools for reading lip movements from video and generating corresponding speech or text, plus systems for syncing audio with lip movements in video. Does NOT include general speech recognition, text-to-speech without visual input, or facial recognition beyond mouth/lip analysis.
There are 9 lip reading synthesis frameworks tracked. 1 score above 50 (established tier). The highest-rated is astorfi/lip-reading-deeplearning at 50/100 with 1,901 stars.
Get all 9 projects as JSON
curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=lip-reading-synthesis&limit=20"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
| # | Framework | Score | Tier |
|---|---|---|---|
| 1 |
astorfi/lip-reading-deeplearning
:unlock: Lip Reading - Cross Audio-Visual Recognition using 3D Architectures |
|
Established |
| 2 |
deepconvolution/LipNet
Automated Lip reading from real-time videos in tensorflow in python |
|
Emerging |
| 3 |
articulateinstruments/DeepLabCut-for-Speech-Production
Trained deep neural-net models for estimating articulatory keypoints from... |
|
Emerging |
| 4 |
MrfoxAK/Evaluate-Lip-reading-using-Deep-Learning-Techniques.
This paper explores Silent Sound Technology, focusing on its potential to... |
|
Experimental |
| 5 |
BenedettoSimone/Lipnet-ITA
LipReadingITA: Keras implementation of the method described in the paper... |
|
Experimental |
| 6 |
Cl0ud-9/Lip-Sync-Video-Generator
An AI-powered pipeline that transforms text into realistic lip-synced... |
|
Experimental |
| 7 |
Viderspace/Look2Listen
End-to-end audio-visual speech enhancement pipeline — from preprocessing to... |
|
Experimental |
| 8 |
mohamed-gasser/Word-Level-LipReading-in-Arabic
Deep Visual Speech Recognition in arabic words |
|
Experimental |
| 9 |
shrvya-shetty/Lip-Reading-DataSet
I have collected my own dataset of lip movements to develop and train the model. |
|
Experimental |