All LLM Inference Engines

145 engines ranked by quality score

Showing 1–100 of 145

#	Engine	Score	Tier	Category	Stars	Language
1	nordwestt/ollama-ai-provider-v2 Vercel AI Provider for running LLMs locally using Ollama	80	Verified	—	99	TypeScript
2	OEvortex/Webscout Webscout is the all-in-one search and AI toolkit you need. Discover insights...	77	Verified	—	336	Python
3	superagentxai/superagentx Move from idea to production in hours with policy-driven autonomous AI...	76	Verified	—	186	Python
4	ArvinLovegood/go-stock 🦄🦄🦄AI赋能股票分析：AI加持的股票分析/选股工具。股票行情获取，AI热点资讯分析，AI资金/财务分析，涨跌报警推送。支持A股，港股，美股。支持市场整体...	75	Verified	—	5,230	Go
5	kubernetes-sigs/lws LeaderWorkerSet: An API for deploying a group of pods as a unit of replication	74	Verified	—	693	Go
6	flashinfer-ai/flashinfer FlashInfer: Kernel Library for LLM Serving	74	Verified	—	5,365	Python
7	ParisNeo/lollms_hub A proxy server for multiple ollama instances with Key security	73	Verified	—	614	HTML
8	Gsync/jobsync JobSync is a self-hosted, open-source job application tracker and AI-powered...	70	Verified	—	503	TypeScript
9	1Panel-dev/1Panel 🔥 1Panel is a modern, open-source VPS control panel — and the only one with...	70	Verified	—	34,884	Go
10	tripolskypetr/agent-swarm-kit A TypeScript library for building orchestrated framework-agnostic...	70	Verified	—	36	TypeScript
11	webgptorg/promptbook Turn your company's scattered knowledge into AI ready Books ✨	69	Established	—	155	TypeScript
12	n4ze3m/page-assist Use your locally running AI models to assist you in your web browsing	69	Established	—	7,756	TypeScript
13	ruvnet/RuVector RuVector is a High Performance, Real-Time, Self-Learning, Vector GNN, Memory...	69	Established	—	3,770	Rust
14	clidey/whodb A lightweight next-gen data explorer - Postgres, MySQL, SQLite, MongoDB,...	68	Established	—	4,713	Go
15	Kochava-Studios/witsy Witsy: desktop AI assistant / universal MCP client	68	Established	—	1,932	TypeScript
16	otwld/ollama-helm Helm chart for Ollama on Kubernetes	67	Established	—	564	Go Template
17	aws-samples/swift-chat A lightning-fast, cross-platform AI Assistant App built with React Native.	67	Established	—	780	TypeScript
18	Jeffser/Alpaca 🦙 Local and online AI hub	65	Established	—	1,486	Python
19	xlite-dev/Awesome-LLM-Inference 📚A curated list of Awesome LLM/VLM Inference Papers with Codes:...	64	Established	—	5,135	Python
20	Azure-Samples/azure-ai-travel-agents A robust enterprise application sample (deployed on ACA) that leverages MCP...	64	Established	—	424	TypeScript
21	pylint-dev/astroid A common base representation of python source code for pylint and other projects	64	Established	—	575	Python
22	openvinotoolkit/openvino_contrib Repository for OpenVINO's extra modules	64	Established	—	171	C++
23	alichherawalla/off-grid-mobile-ai The Swiss Army Knife of Offline AI. Chat, Speak, and Generate Images -...	64	Established	—	1,444	TypeScript
24	sebastienrousseau/bankstatementparser Parse bank statements across CAMT, PAIN.001, CSV, OFX/QFX, MT940, and PDFs...	63	Established	—	13	Python
25	kac89/vulnrepo VULNRΞPO - Free vulnerability report generator and repository, end-to-end...	63	Established	—	551	TypeScript
26	AsyncFuncAI/deepwiki-open Open Source DeepWiki: AI-Powered Wiki Generator for GitHub/Gitlab/Bitbucket...	62	Established	—	15,551	Python
27	control-theory/gonzo Gonzo! The Go based TUI log analysis tool	60	Established	—	2,571	Go
28	PlanExeOrg/PlanExe Create a plan from a description in minutes	59	Established	—	365	Python
29	anirbanbasu/ollama-downloader A library and Hugging Face model downloader for Ollama.	58	Established	—	5	Python
30	cogwheel0/conduit Mobile (iOS/Android) client for OpenWebUI. Chat with your self‑hosted AI.	58	Established	—	1,298	Dart
31	abhibambhaniya/GenZ-LLM-Analyzer LLM Inference analyzer for different hardware platforms	58	Established	—	109	Jupyter Notebook
32	0xdea/oneiromancer Reverse engineering assistant that uses a locally running LLM to aid with...	56	Established	—	129	Rust
33	jarvesusaram99/Portable-AI-USB A 100% offline, fully portable, zero-trace AI (Ollama + Llama 3 +...	56	Established	—	328	Shell
34	aws-samples/sample-genai-on-eks-starter-kit A comprehensive toolkit for deploying production-ready Generative AI...	56	Established	—	51	JavaScript
35	SJTU-DENG-Lab/Diffulex Flexible and Pluggable Serving Engine for Diffusion LLMs	54	Established	—	68	Python
36	jenkinsci/explain-error-plugin Explain Jenkins job failures with AI	54	Established	—	31	Java
37	alvarobartt/hf-mem A CLI to estimate inference memory requirements for Hugging Face models,...	54	Established	—	891	Python
38	thushan/olla High-performance lightweight proxy and load balancer for LLM infrastructure....	53	Established	—	189	Go
39	lastsunday/job-hunting 职位猎人 - 协助找工作的工具	53	Established	—	345	JavaScript
40	LettuceAI/app Privacy-first AI roleplay & storytelling app with long-term memory, custom...	53	Established	—	47	TypeScript
41	sajanv88/full_stack_fastapi_react_template About Full stack, modern multi-tenancy web template. Using FastAPI, React,...	52	Established	—	35	TypeScript
42	Epistates/pmetal Powdered Metal — High performance LLM fine-tuning framework for Apple Silicon	52	Established	—	218	Rust
43	mnvsk97/eyeroll AI eyes that roll through video footage — watch, understand, act	50	Established	—	8	Python
44	SethRobinson/UGTLive Live AI-powered screen translation via LLMs & GPU OCR. 26 languages, manga...	50	Established	—	73	C#
45	ulfurinn/wongi-engine-elixir A rule engine written in Elixir.	48	Emerging	—	30	Elixir
46	frumu-ai/tandem Your AI coworker for any folder: local-first, secure by design,...	48	Emerging	—	88	Rust
47	Mybono/ai-orchestrator Portable multi-agent AI developer setup for Claude Code + Ollama. Role-based...	48	Emerging	—	85	Shell
48	SirUli/homeassistant-ollama-addon Provides an Home Assistant addon for Ollama	48	Emerging	—	71	—
49	lukeslp/alt-text-local-llm Local tool for generating descriptive alt text for images using vision models	48	Emerging	—	13	JavaScript
50	bartvantuijn/noton A free and open documentation platform built with Laravel and Filament,...	47	Emerging	—	140	PHP
51	christopherkarani/Conduit 🦑 Unified Swift SDK for LLM inference across local and cloud providers	47	Emerging	—	83	Swift
52	omarluq/cc-relay ⚡️ Blazing fast LLMs API Gateway written in Go	47	Emerging	—	68	Go
53	gty111/gLLM gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM...	46	Emerging	—	56	Python
54	Romfos/AutoTests.Framework BddDotNet based autotest framework with Playwright integration for .NET	46	Emerging	—	7	C#
55	georgiosnikitas/brain-break 🧠🔨 Brain Break is an AI-powered terminal quiz app.	46	Emerging	—	11	TypeScript
56	xieguigang/LLMs A small LLM toolkit written in VisualBasic	46	Emerging	—	3	HTML
57	josephgoksu/TaskWing Local-first AI knowledge layer. Extract architecture, query from any AI tool...	45	Emerging	—	85	Go
58	shadoq/refio Refio – open source, local-first coding companion for IntelliJ	45	Emerging	—	5	Kotlin
59	Olawoyin007/empathySync Help that knows when to stop.	45	Emerging	—	6	Python
60	B-A-M-N/SOLLOL Super Ollama Load Balancer - Performance-aware routing for distributed...	44	Emerging	—	4	Python
61	The-Focus-AI/umwelten CLI tool for evaluating and comparing AI models across Google, Ollama,...	44	Emerging	—	3	TypeScript
62	avikeid2007/KaiROS-AI KaiROS AI— Intelligence, Precisely When It Matters.	44	Emerging	—	9	C#
63	lalomorales22/myBay easily publish your products to your ebay sellers account (takes a little setting up)	44	Emerging	—	26	Python
64	geeks-accelerator/ollama-herd Smart inference router that herds your Ollama instances into one endpoint....	44	Emerging	—	7	Python
65	Kyaw-Min-Thant/plux Plux: AI-powered filetree that lets you grab files with one click and save...	44	Emerging	—	5	TypeScript
66	sharktide/inferenceport-ai Run powerful language models locally — privately. InferencePort AI makes it...	44	Emerging	—	4	TypeScript
67	EduardozinYT/ai-instagram-organizer 📸 Organize your Instagram posts effortlessly with AI, generating smart...	44	Emerging	—	3	Python
68	samestrin/llm-env Force any OpenAI-compatible tool (Aider, Fabric, Interpreter) to use Gemini,...	43	Emerging	—	4	Shell
69	uwuclxdy/ollama-lmstudio-proxy Access LM Studio through Ollama API.	43	Emerging	—	11	Rust
70	AJ/FinSight Privacy focused bank and credit card statement insights and visualizer	43	Emerging	—	8	JavaScript
71	OpenDCAI/Flash-MinerU Ray-powered accelerator for MinerU, turning PDF → Markdown into a scalable,...	43	Emerging	—	43	Python
72	DeepLink-org/Pulsing Pulsing is a distributed actor framework that provides a communication...	43	Emerging	—	7	Rust
73	DylanLRPollock/Monkey-Head-Project Huey is a prototype robotic A.I./O.S.	42	Emerging	—	7	Python
74	wink-wink-wink555/blind_navigation Travel Assistance System for the Visually Impaired (ARIADNE) is an...	42	Emerging	—	27	HTML
75	B-A-M-N/Sheppard Sheppard is an AI agent for Ollama, handling memory, automation, and...	42	Emerging	—	7	Python
76	arrase/ollama-agent Ollama Agent is a powerful command-line tool (CLI and REPL) that allows you...	41	Emerging	—	4	Python
77	Lourdle/cosyvoice.cpp CosyVoice inference in C/C++	41	Emerging	—	5	C++
78	dxiv/dxa-deimos A terminal AI coding agent that works across local or cloud models.	41	Emerging	—	6	TypeScript
79	urmzd/incipit Here begins the new career. A template-driven CLI that transforms structured...	41	Emerging	—	11	Go
80	BigBill1418/DroneOpsCommand Self-hosted mission management platform for commercial drone operators. AI...	40	Emerging	—	7	TypeScript
81	anomixer/openclaw-setup OpenClaw + Ollama + Telegram Quick Setup Guide \| 快速安裝教學 (also see...	40	Emerging	—	39	JavaScript
82	DozaVisuals/doza-assist Local-first AI-powered interview transcription and editing tool for...	40	Emerging	—	7	HTML
83	siphalion/quisium Python library that sits between your app and the LLM client, adding...	40	Emerging	—	4	Python
84	djethino/UnityGameTranslator Universal Unity games automatic local AI translation and communtity sharing...	39	Emerging	—	7	C#
85	kkomelin/n8n-ollama-desktop LonelyNathan: n8n + Ollama Desktop App	39	Emerging	—	6	TypeScript
86	hoornet/home-mind AI assistant for Home Assistant with cognitive memory. Supports Anthropic,...	39	Emerging	—	52	TypeScript
87	chetstriker/LLMtary Autonomous AI-powered penetration testing platform. LLM-driven recon,...	39	Emerging	—	7	Dart
88	apimeister/gguf-runner a gguf model runner	39	Emerging	—	4	Rust
89	joli-sys/KlustrEye KlustrEye — Native desktop Kubernetes IDE built with Tauri, React & Rust....	39	Emerging	—	3	TypeScript
90	zoott28354/ai_assistant Local desktop AI assistant for screenshots, translations, image analysis and...	39	Emerging	—	3	Python
91	Rahul-14507/MELLM Lightweight Modular AI Routing Engine for Local LLMs — Run specialised...	38	Emerging	—	31	Python
92	linux4life1/front-porch-AI GPL 3 replacement of defunct Backyard AI character creation/chat platform	38	Emerging	—	22	Dart
93	tmseidel/ai-gitea-bot An application that adds Bot-capabilities to a Gitea instance and works with...	38	Emerging	—	4	Java
94	AiursoftWeb/OllamaGateway Ollama Gateway: Supercharge your native Ollama with enterprise-grade API...	38	Emerging	—	5	C#
95	openziti/llm-gateway Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and...	37	Emerging	—	49	Go
96	ondeinference/onde-swift Onde Inference Swift SDK	36	Emerging	—	6	Swift
97	AriesOxO/piz 智能终端命令助手 — 用自然语言生成 Shell 命令，支持 OpenAI/Claude/Gemini/Ollama 等多后端	35	Emerging	—	36	Rust
98	KERNlang/kern Universal IR compiler — .kern files compile to Next.js, Vue, Express,...	35	Emerging	—	7	TypeScript
99	Picovoice/llm-compression-benchmark LLM Compression Benchmark	35	Emerging	—	22	Python
100	k-zehnder/gophersignal Gopher Signal uses smart technology to quickly summarize important points...	35	Emerging	—	26	TypeScript

1 2 Next »