toverainc/willow-inference-server
Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS
This project helps you turn spoken audio into text and generate speech from text using your own computer hardware. It takes audio input (like a voice recording or live speech) and produces transcribed text or takes text input and generates natural-sounding speech. This is for technical users or system administrators who want to host their own robust speech recognition and text-to-speech services locally, rather than relying on cloud providers.
495 stars.
Use this if you need a high-performance, self-hosted solution for real-time audio transcription and speech generation, especially for applications like voice assistants or accessibility tools, and have compatible GPU hardware.
Not ideal if you're looking for a simple, plug-and-play desktop application for occasional speech tasks or do not have the technical expertise to set up and manage a server.
Stars
495
Forks
58
Language
Python
License
Apache-2.0
Category
Last pushed
Feb 12, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/toverainc/willow-inference-server"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related models
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
alibaba/MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering...
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source,...
tensorzero/tensorzero
TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM...