Speech Synthesis Diffusion Diffusion Models

Diffusion models for speech and audio generation including TTS, voice conversion, singing synthesis, and vocoding. Does NOT include general image diffusion, music generation without speech focus, or non-diffusion audio processing.

There are 55 speech synthesis diffusion models tracked. 2 score above 50 (established tier). The highest-rated is PrunaAI/pruna at 63/100 with 1,142 stars. 1 of the top 10 are actively maintained.

Get all 55 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=diffusion&subcategory=speech-synthesis-diffusion&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Model	Score	Tier	Stars	Language
1	PrunaAI/pruna Pruna is a model optimization framework built for developers, enabling you...	63	Established	1,142	Python
2	bytedance/LatentSync Taming Stable Diffusion for Lip Sync!	51	Established	5,506	Python
3	haoheliu/AudioLDM-training-finetuning AudioLDM training, finetuning, evaluation and inference.	48	Emerging	297	Python
4	Text-to-Audio/Make-An-Audio PyTorch Implementation of Make-An-Audio (ICML'23) with a Text-to-Audio...	47	Emerging	669	Python
5	teticio/audio-diffusion Apply diffusion models using the new Hugging Face diffusers package to...	44	Emerging	789	Jupyter Notebook
6	ivanvovk/WaveGrad Implementation of WaveGrad high-fidelity vocoder from Google Brain in PyTorch.	44	Emerging	408	Jupyter Notebook
7	Rongjiehuang/ProDiff PyTorch Implementation of ProDiff (ACM-MM'22) with a Extremely-Fast...	44	Emerging	432	Python
8	keonlee9420/DiffSinger PyTorch implementation of DiffSinger: Singing Voice Synthesis via Shallow...	44	Emerging	247	Python
9	keonlee9420/DiffGAN-TTS PyTorch Implementation of DiffGAN-TTS: High-Fidelity and Efficient...	43	Emerging	347	Python
10	sayakpaul/diffusers-torchao End-to-end recipes for optimizing diffusion models with torchao and...	42	Emerging	397	Python
11	Aratako/Irodori-TTS A Flow Matching-based Text-to-Speech Model with Emoji-driven Style Control	42	Emerging	40	Python
12	yochaiye/LipVoicer Official Code implementation for the ICLR paper "LipVoicer: Generating...	41	Emerging	86	Python
13	segmind/distill-sd Segmind Distilled diffusion	40	Emerging	619	Python
14	zhenye234/CoMoSpeech ACM MM 2023 CoMoSpeech: One-Step Speech and Singing Voice Synthesis via...	40	Emerging	213	Python
15	huggingface/diffusion-fast Faster generation with text-to-image diffusion models.	39	Emerging	232	Python
16	sony/soundctm Pytorch implementation of SoundCTM	37	Emerging	101	Python
17	trinhtuanvubk/Diff-VC Diffusion Model for Voice Conversion	37	Emerging	69	Jupyter Notebook
18	G-U-N/Phased-Consistency-Model [NeurIPS 2024] Boosting the performance of consistency models with PCM!	37	Emerging	514	Python
19	junhsss/consistency-models A Toolkit for OpenAI's Consistency Models.	37	Emerging	207	Python
20	xandergos/sCM-mnist Unofficial implementation of "Simplifying, Stabilizing & Scaling...	36	Emerging	89	Python
21	mazumdarsoumya/TempoSyncDiff Few-step diffusion for audio-driven talking head generation making diffusion...	35	Emerging	2	Python
22	TencentARC/AudioStory AudioStory: Generating Long-Form Narrative Audio with Large Language Models	32	Emerging	299	Jupyter Notebook
23	FireRedTeam/Target-Driven-Distillation Consistency Distillation with Target Timestep Selection and Decoupled Guidance	32	Emerging	104	Python
24	koichi-saito-sony/soundctm_dit_iclr Pytorch implementation of SoundCTM-DiT	31	Emerging	4	Jupyter Notebook
25	JiauZhang/binary-latent-diffusion Implementation of Binary Latent Diffusion	31	Emerging	51	Python
26	hayeong0/Diff-HierVC Official Pytorch Implementation of "Diff-HierVC: Diffusion-based...	31	Emerging	235	Python
27	0x7o/DeepMozart Audio generation using diffusion models	31	Emerging	2	Python
28	mbreuss/consistency_models_toy_task Unofficial minimal implementation of consistency models (CM) proposed by...	30	Emerging	21	Python
29	MirageML/MirageStock Open-Source Implementations of Multi-Modal Diffusion Models Optimized for...	30	Emerging	198	Python
30	ashutosh1919/consistency-models Ready to run PyTorch implementation of Consistency Models: One-Step Image...	30	Emerging	6	Shell
31	OpenGVLab/LORIS [ICML2023] Long-Term Rhythmic Video Soundtracker	29	Experimental	62	Python
32	seahore/PPG-GradVC A diffusion-based cross-lingual voice conversion model, as my bachelor's thesis	29	Experimental	44	Python
33	drakyanerlanggarizkiwardhana/Diffusers 🤗 Diffusers: State-of-the-art diffusion models for image and audio...	29	Experimental	1	Python
34	jabir-zheng/TCD Official Repository of the paper "Trajectory Consistency Distillation"	28	Experimental	363	Python
35	smsharma/consistency-models Implementation of Consistency Models (Song et al 2023) for few-step image...	27	Experimental	19	Jupyter Notebook
36	Consistency-TTA/consistency-tta.github.io Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation	26	Experimental	7	HTML
37	AxiumCrisis61/StableSVC StableSVC: Latent Diffusion Model for Singing Voice Conversion (originally...	23	Experimental	4	Python
38	testzer0/GradTTS-unoffical My unofficial implementation of Grad-TTS (ICML 2021)	23	Experimental	4	Jupyter Notebook
39	Bai-YT/ConsistencyTTA ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with...	23	Experimental	39	Python
40	romanycc/Audio-Diffusion Audio Diffusion	23	Experimental	4	Python
41	LiangXu123/Robust-One-step-Speech-Enhancement-via-Consistency-Distillation-ROSE-CD- Robust One-step Speech Enhancement via Consistency Distillation...	22	Experimental	10	—
42	mbreuss/consistency_trajectory_models_toy_task Minimal unofficial implementation of Consistency Trajectory models on a 1D toy task.	22	Experimental	22	Python
43	juanalonso/diffusion-audio Lista de modelos y aplicaciones basadas en diffusion	20	Experimental	11	—
44	slegroux/nimrod minimal deep learning framework	20	Experimental	2	Jupyter Notebook
45	quickgrid/distill-sd Experiment with latent diffusion models.	19	Experimental	3	Python
46	minyoungpark1/Speech-Enhancement Unofficial implementation of SCP-GAN	19	Experimental	18	Python
47	jwliao1209/DiffMusic 🎼 DiffMusic: A Training-Free Diffusion Framework for Music Inverse Problem	19	Experimental	4	Python
48	instill-ai/model-diffusion-dvc ⚗️ Diffusion model repository based on HuggingFace Diffusion 2.1 managed by DVC	15	Experimental	2	Python
49	michalsvento/UnNAFx Supplementary code for paper submitted to DAFx 2025	13	Experimental	4	Python
50	Jason-cs18/HetServe-Foundation A Overview of Efficiently Serving Foundation Models across Edge Devices	13	Experimental	14	—
51	Shiying-Zhang/-diffusion-model-genealogy 🧬 Diffusion Model Genealogy - Mapping the family relationships between...	12	Experimental	1	—
52	XinleiNIU/SoundMorpher This is implementation code for "SoundMorpher: Perceptually-Uniform Sound...	12	Experimental	5	Jupyter Notebook
53	7-4-7/BirdGen Implementation of classifier guided diiffusion model on a procedurally...	11	Experimental	—	Jupyter Notebook
54	manthan89-py/OpenSource-Diffusion-Models-Experiment This repo analyzes Open Source Diffusion models for generating...	11	Experimental	3	Jupyter Notebook
55	VladimirZelenokor1/ML-Project---Voice-Conversion-with-Diffusion-Models Project on real time voice conversion with diffusion models	10	Experimental	1	Python

Comparisons in this category

DiffSinger and DiffGAN-TTS (44 vs 43)