The LLM Inference Directory
Quality-scored directory of 145 llm inference engines, updated daily. Every engine scored on maintenance, adoption, maturity, and community signals.
LLM inference engines for self-hosted model serving — vLLM, TGI, SGLang, Ollama, llama.cpp, and the tools that make local inference fast and efficient.
10
70–100
34
50–69
69
30–49
32
10–29
Top engines by quality score
| # | Engine | Score |
|---|---|---|
| 1 |
nordwestt/ollama-ai-provider-v2
Vercel AI Provider for running LLMs locally using Ollama |
|
| 2 |
OEvortex/Webscout
Webscout is the all-in-one search and AI toolkit you need. Discover insights... |
|
| 3 |
superagentxai/superagentx
Move from idea to production in hours with policy-driven autonomous AI... |
|
| 4 |
ArvinLovegood/go-stock
🦄🦄🦄AI赋能股票分析：AI加持的股票分析/选股工具。股票行情获取，AI热点资讯分析，AI资金/财务分析，涨跌报警推送。支持A股，港股，美股。支持市场整体... |
|
| 5 |
kubernetes-sigs/lws
LeaderWorkerSet: An API for deploying a group of pods as a unit of replication |
|
| 6 |
flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving |
|
| 7 |
ParisNeo/lollms_hub
A proxy server for multiple ollama instances with Key security |
|
| 8 |
Gsync/jobsync
JobSync is a self-hosted, open-source job application tracker and AI-powered... |
|
| 9 |
1Panel-dev/1Panel
🔥 1Panel is a modern, open-source VPS control panel — and the only one with... |
|
| 10 |
tripolskypetr/agent-swarm-kit
A TypeScript library for building orchestrated framework-agnostic... |
|
| 11 |
webgptorg/promptbook
Turn your company's scattered knowledge into AI ready Books ✨ |
|
| 12 |
n4ze3m/page-assist
Use your locally running AI models to assist you in your web browsing |
|
| 13 |
ruvnet/RuVector
RuVector is a High Performance, Real-Time, Self-Learning, Vector GNN, Memory... |
|
| 14 |
clidey/whodb
A lightweight next-gen data explorer - Postgres, MySQL, SQLite, MongoDB,... |
|
| 15 |
Kochava-Studios/witsy
Witsy: desktop AI assistant / universal MCP client |
|
| 16 |
otwld/ollama-helm
Helm chart for Ollama on Kubernetes |
|
| 17 |
aws-samples/swift-chat
A lightning-fast, cross-platform AI Assistant App built with React Native. |
|
| 18 |
Jeffser/Alpaca
🦙 Local and online AI hub |
|
| 19 |
xlite-dev/Awesome-LLM-Inference
📚A curated list of Awesome LLM/VLM Inference Papers with Codes:... |
|
| 20 |
Azure-Samples/azure-ai-travel-agents
A robust enterprise application sample (deployed on ACA) that leverages MCP... |
|