Uncategorized LLM Inference Engines

There are 145 uncategorized engines tracked. 10 score above 70 (verified tier). The highest-rated is nordwestt/ollama-ai-provider-v2 at 80/100 with 99 stars and 857,709 monthly downloads. 6 of the top 10 are actively maintained.

Get all 145 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=llm-inference&subcategory=uncategorized&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Engine	Score	Tier	Stars	Language
1	nordwestt/ollama-ai-provider-v2 Vercel AI Provider for running LLMs locally using Ollama	80	Verified	99	TypeScript
2	OEvortex/Webscout Webscout is the all-in-one search and AI toolkit you need. Discover insights...	77	Verified	336	Python
3	superagentxai/superagentx Move from idea to production in hours with policy-driven autonomous AI...	76	Verified	186	Python
4	ArvinLovegood/go-stock 🦄🦄🦄AI赋能股票分析：AI加持的股票分析/选股工具。股票行情获取，AI热点资讯分析，AI资金/财务分析，涨跌报警推送。支持A股，港股，美股。支持市场整体...	75	Verified	5,230	Go
5	kubernetes-sigs/lws LeaderWorkerSet: An API for deploying a group of pods as a unit of replication	74	Verified	693	Go
6	flashinfer-ai/flashinfer FlashInfer: Kernel Library for LLM Serving	74	Verified	5,365	Python
7	ParisNeo/lollms_hub A proxy server for multiple ollama instances with Key security	73	Verified	614	HTML
8	Gsync/jobsync JobSync is a self-hosted, open-source job application tracker and AI-powered...	70	Verified	503	TypeScript
9	1Panel-dev/1Panel 🔥 1Panel is a modern, open-source VPS control panel — and the only one with...	70	Verified	34,884	Go
10	tripolskypetr/agent-swarm-kit A TypeScript library for building orchestrated framework-agnostic...	70	Verified	36	TypeScript
11	webgptorg/promptbook Turn your company's scattered knowledge into AI ready Books ✨	69	Established	155	TypeScript
12	n4ze3m/page-assist Use your locally running AI models to assist you in your web browsing	69	Established	7,756	TypeScript
13	ruvnet/RuVector RuVector is a High Performance, Real-Time, Self-Learning, Vector GNN, Memory...	69	Established	3,770	Rust
14	clidey/whodb A lightweight next-gen data explorer - Postgres, MySQL, SQLite, MongoDB,...	68	Established	4,713	Go
15	Kochava-Studios/witsy Witsy: desktop AI assistant / universal MCP client	68	Established	1,932	TypeScript
16	otwld/ollama-helm Helm chart for Ollama on Kubernetes	67	Established	564	Go Template
17	aws-samples/swift-chat A lightning-fast, cross-platform AI Assistant App built with React Native.	67	Established	780	TypeScript
18	Jeffser/Alpaca 🦙 Local and online AI hub	65	Established	1,486	Python
19	xlite-dev/Awesome-LLM-Inference 📚A curated list of Awesome LLM/VLM Inference Papers with Codes:...	64	Established	5,135	Python
20	Azure-Samples/azure-ai-travel-agents A robust enterprise application sample (deployed on ACA) that leverages MCP...	64	Established	424	TypeScript
21	pylint-dev/astroid A common base representation of python source code for pylint and other projects	64	Established	575	Python
22	openvinotoolkit/openvino_contrib Repository for OpenVINO's extra modules	64	Established	171	C++
23	alichherawalla/off-grid-mobile-ai The Swiss Army Knife of Offline AI. Chat, Speak, and Generate Images -...	64	Established	1,444	TypeScript
24	sebastienrousseau/bankstatementparser Parse bank statements across CAMT, PAIN.001, CSV, OFX/QFX, MT940, and PDFs...	63	Established	13	Python
25	kac89/vulnrepo VULNRΞPO - Free vulnerability report generator and repository, end-to-end...	63	Established	551	TypeScript
26	AsyncFuncAI/deepwiki-open Open Source DeepWiki: AI-Powered Wiki Generator for GitHub/Gitlab/Bitbucket...	62	Established	15,551	Python
27	control-theory/gonzo Gonzo! The Go based TUI log analysis tool	60	Established	2,571	Go
28	PlanExeOrg/PlanExe Create a plan from a description in minutes	59	Established	365	Python
29	anirbanbasu/ollama-downloader A library and Hugging Face model downloader for Ollama.	58	Established	5	Python
30	cogwheel0/conduit Mobile (iOS/Android) client for OpenWebUI. Chat with your self‑hosted AI.	58	Established	1,298	Dart
31	abhibambhaniya/GenZ-LLM-Analyzer LLM Inference analyzer for different hardware platforms	58	Established	109	Jupyter Notebook
32	0xdea/oneiromancer Reverse engineering assistant that uses a locally running LLM to aid with...	56	Established	129	Rust
33	jarvesusaram99/Portable-AI-USB A 100% offline, fully portable, zero-trace AI (Ollama + Llama 3 +...	56	Established	328	Shell
34	aws-samples/sample-genai-on-eks-starter-kit A comprehensive toolkit for deploying production-ready Generative AI...	56	Established	51	JavaScript
35	SJTU-DENG-Lab/Diffulex Flexible and Pluggable Serving Engine for Diffusion LLMs	54	Established	68	Python
36	jenkinsci/explain-error-plugin Explain Jenkins job failures with AI	54	Established	31	Java
37	alvarobartt/hf-mem A CLI to estimate inference memory requirements for Hugging Face models,...	54	Established	891	Python
38	thushan/olla High-performance lightweight proxy and load balancer for LLM infrastructure....	53	Established	189	Go
39	lastsunday/job-hunting 职位猎人 - 协助找工作的工具	53	Established	345	JavaScript
40	LettuceAI/app Privacy-first AI roleplay & storytelling app with long-term memory, custom...	53	Established	47	TypeScript
41	sajanv88/full_stack_fastapi_react_template About Full stack, modern multi-tenancy web template. Using FastAPI, React,...	52	Established	35	TypeScript
42	Epistates/pmetal Powdered Metal — High performance LLM fine-tuning framework for Apple Silicon	52	Established	218	Rust
43	mnvsk97/eyeroll AI eyes that roll through video footage — watch, understand, act	50	Established	8	Python
44	SethRobinson/UGTLive Live AI-powered screen translation via LLMs & GPU OCR. 26 languages, manga...	50	Established	73	C#
45	ulfurinn/wongi-engine-elixir A rule engine written in Elixir.	48	Emerging	30	Elixir
46	frumu-ai/tandem Your AI coworker for any folder: local-first, secure by design,...	48	Emerging	88	Rust
47	Mybono/ai-orchestrator Portable multi-agent AI developer setup for Claude Code + Ollama. Role-based...	48	Emerging	85	Shell
48	SirUli/homeassistant-ollama-addon Provides an Home Assistant addon for Ollama	48	Emerging	71	—
49	lukeslp/alt-text-local-llm Local tool for generating descriptive alt text for images using vision models	48	Emerging	13	JavaScript
50	bartvantuijn/noton A free and open documentation platform built with Laravel and Filament,...	47	Emerging	140	PHP
51	christopherkarani/Conduit 🦑 Unified Swift SDK for LLM inference across local and cloud providers	47	Emerging	83	Swift
52	omarluq/cc-relay ⚡️ Blazing fast LLMs API Gateway written in Go	47	Emerging	68	Go
53	gty111/gLLM gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM...	46	Emerging	56	Python
54	Romfos/AutoTests.Framework BddDotNet based autotest framework with Playwright integration for .NET	46	Emerging	7	C#
55	georgiosnikitas/brain-break 🧠🔨 Brain Break is an AI-powered terminal quiz app.	46	Emerging	11	TypeScript
56	xieguigang/LLMs A small LLM toolkit written in VisualBasic	46	Emerging	3	HTML
57	josephgoksu/TaskWing Local-first AI knowledge layer. Extract architecture, query from any AI tool...	45	Emerging	85	Go
58	shadoq/refio Refio – open source, local-first coding companion for IntelliJ	45	Emerging	5	Kotlin
59	Olawoyin007/empathySync Help that knows when to stop.	45	Emerging	6	Python
60	B-A-M-N/SOLLOL Super Ollama Load Balancer - Performance-aware routing for distributed...	44	Emerging	4	Python
61	The-Focus-AI/umwelten CLI tool for evaluating and comparing AI models across Google, Ollama,...	44	Emerging	3	TypeScript
62	avikeid2007/KaiROS-AI KaiROS AI— Intelligence, Precisely When It Matters.	44	Emerging	9	C#
63	lalomorales22/myBay easily publish your products to your ebay sellers account (takes a little setting up)	44	Emerging	26	Python
64	geeks-accelerator/ollama-herd Smart inference router that herds your Ollama instances into one endpoint....	44	Emerging	7	Python
65	Kyaw-Min-Thant/plux Plux: AI-powered filetree that lets you grab files with one click and save...	44	Emerging	5	TypeScript
66	sharktide/inferenceport-ai Run powerful language models locally — privately. InferencePort AI makes it...	44	Emerging	4	TypeScript
67	EduardozinYT/ai-instagram-organizer 📸 Organize your Instagram posts effortlessly with AI, generating smart...	44	Emerging	3	Python
68	samestrin/llm-env Force any OpenAI-compatible tool (Aider, Fabric, Interpreter) to use Gemini,...	43	Emerging	4	Shell
69	uwuclxdy/ollama-lmstudio-proxy Access LM Studio through Ollama API.	43	Emerging	11	Rust
70	AJ/FinSight Privacy focused bank and credit card statement insights and visualizer	43	Emerging	8	JavaScript
71	OpenDCAI/Flash-MinerU Ray-powered accelerator for MinerU, turning PDF → Markdown into a scalable,...	43	Emerging	43	Python
72	DeepLink-org/Pulsing Pulsing is a distributed actor framework that provides a communication...	43	Emerging	7	Rust
73	DylanLRPollock/Monkey-Head-Project Huey is a prototype robotic A.I./O.S.	42	Emerging	7	Python
74	wink-wink-wink555/blind_navigation Travel Assistance System for the Visually Impaired (ARIADNE) is an...	42	Emerging	27	HTML
75	B-A-M-N/Sheppard Sheppard is an AI agent for Ollama, handling memory, automation, and...	42	Emerging	7	Python
76	arrase/ollama-agent Ollama Agent is a powerful command-line tool (CLI and REPL) that allows you...	41	Emerging	4	Python
77	Lourdle/cosyvoice.cpp CosyVoice inference in C/C++	41	Emerging	5	C++
78	dxiv/dxa-deimos A terminal AI coding agent that works across local or cloud models.	41	Emerging	6	TypeScript
79	urmzd/incipit Here begins the new career. A template-driven CLI that transforms structured...	41	Emerging	11	Go
80	BigBill1418/DroneOpsCommand Self-hosted mission management platform for commercial drone operators. AI...	40	Emerging	7	TypeScript
81	anomixer/openclaw-setup OpenClaw + Ollama + Telegram Quick Setup Guide \| 快速安裝教學 (also see...	40	Emerging	39	JavaScript
82	DozaVisuals/doza-assist Local-first AI-powered interview transcription and editing tool for...	40	Emerging	7	HTML
83	siphalion/quisium Python library that sits between your app and the LLM client, adding...	40	Emerging	4	Python
84	djethino/UnityGameTranslator Universal Unity games automatic local AI translation and communtity sharing...	39	Emerging	7	C#
85	kkomelin/n8n-ollama-desktop LonelyNathan: n8n + Ollama Desktop App	39	Emerging	6	TypeScript
86	hoornet/home-mind AI assistant for Home Assistant with cognitive memory. Supports Anthropic,...	39	Emerging	52	TypeScript
87	chetstriker/LLMtary Autonomous AI-powered penetration testing platform. LLM-driven recon,...	39	Emerging	7	Dart
88	apimeister/gguf-runner a gguf model runner	39	Emerging	4	Rust
89	joli-sys/KlustrEye KlustrEye — Native desktop Kubernetes IDE built with Tauri, React & Rust....	39	Emerging	3	TypeScript
90	zoott28354/ai_assistant Local desktop AI assistant for screenshots, translations, image analysis and...	39	Emerging	3	Python
91	Rahul-14507/MELLM Lightweight Modular AI Routing Engine for Local LLMs — Run specialised...	38	Emerging	31	Python
92	linux4life1/front-porch-AI GPL 3 replacement of defunct Backyard AI character creation/chat platform	38	Emerging	22	Dart
93	tmseidel/ai-gitea-bot An application that adds Bot-capabilities to a Gitea instance and works with...	38	Emerging	4	Java
94	AiursoftWeb/OllamaGateway Ollama Gateway: Supercharge your native Ollama with enterprise-grade API...	38	Emerging	5	C#
95	openziti/llm-gateway Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and...	37	Emerging	49	Go
96	ondeinference/onde-swift Onde Inference Swift SDK	36	Emerging	6	Swift
97	AriesOxO/piz 智能终端命令助手 — 用自然语言生成 Shell 命令，支持 OpenAI/Claude/Gemini/Ollama 等多后端	35	Emerging	36	Rust
98	KERNlang/kern Universal IR compiler — .kern files compile to Next.js, Vue, Express,...	35	Emerging	7	TypeScript
99	Picovoice/llm-compression-benchmark LLM Compression Benchmark	35	Emerging	22	Python
100	k-zehnder/gophersignal Gopher Signal uses smart technology to quickly summarize important points...	35	Emerging	26	TypeScript
101	vijaytank/eclipsellama Privacy-first AI coding assistant for Eclipse (Ollama)	35	Emerging	4	Java
102	chocks/locode Local-first AI coding CLI. Routes simple tasks to a local LLM (Ollama),...	34	Emerging	23	TypeScript
103	ARPAHLS/micro-f1-mask Zero-Latency PII Scrubbing Middleware Agent for Enterprise Cloud Compliance.	34	Emerging	10	Python
104	Entropic-Science/qr-sampler Integrate any source of randomness into LLM token sampling. Easily create...	33	Emerging	13	Python
105	mlco2/ecologits-api 🌱 Estimate the environmental footprint of GenAI with EcoLogits API	32	Emerging	4	Python
106	0xMMA/KeyLint Fix text instantly with AI — one hotkey, perfect writing. Supports OpenAI,...	32	Emerging	4	TypeScript
107	thenetcircle/logistik Machine learning logistics and serving platform	32	Emerging	4	JavaScript
108	argeo/argeo-jjml Java bindings for the ggml family of machine learning libraries, especially...	32	Emerging	4	Java
109	whereclaw/whereclaw OpenClaw one-click installation & Running local models	32	Emerging	3	Rust
110	Resk-Security/resk-llm-ts A comprehensive security toolkit for JavaScript applications using Large...	32	Emerging	7	TypeScript
111	upioneer/homelab Collection of apps and utilities for automation and homelab	31	Emerging	3	—
112	PrzemyslawSwiderski/python-uv-gradle-plugin Gradle Plugin for Python projects using uv tool	30	Emerging	6	Kotlin
113	MekayelAnik/vllm-cpu Wheels & Docker images for running vLLM on CPU-only systems, optimized for...	30	Emerging	6	Shell
114	clanker-lover/codedocent Code visualization for non-programmers. A docent for your codebase.	29	Experimental	9	Python
115	crstian19/llamit A VS Code extension to generate semantic commit messages using your local...	29	Experimental	9	TypeScript
116	JoelHJames1/Nexus-Inference-Engine- NEXUS: Production C++ inference engine for Apple Silicon. Run 400B+ LLMs on...	29	Experimental	3	C++
117	ascarola/verdictmail AI-powered email threat analysis daemon — IMAP IDLE monitoring,...	28	Experimental	5	Python
118	PeterVanPercson/buildcored-orcas A 30-day build challenge for people who can execute under pressure. You will...	27	Experimental	36	Python
119	Sagargupta16/ai-git-hooks AI-powered git hooks - auto-review diffs, generate commit messages, scan for...	27	Experimental	3	Shell
120	Saisathvik94/codemaxx CodeMaxx – AI-powered tool to explain, fix, and improve your code. CodeMaxx...	27	Experimental	3	Go
121	boxxy-dev/boxxy Stupid Linux Terminal	27	Experimental	66	Rust
122	Olawoyin007/intentKeeper A digital bodyguard for your mind - local-first content filter that...	27	Experimental	4	Python
123	AlexHettle/MeadowPy Beginner-friendly Python IDE with local AI features, teaching tools,...	27	Experimental	3	Python
124	SysAdminDoc/project-nomad-desktop Cross-platform desktop edition fork of Project N.O.M.A.D with more features...	27	Experimental	9	JavaScript
125	wshm-dev/wshm AI-powered repository agent for GitHub, GitLab, Gitea, Azure DevOps. Issue...	27	Experimental	11	Rust
126	msaad00/cloud-security Closed-loop cloud security benchmarks and remediation — 5 skills, 114 tests,...	27	Experimental	3	Python
127	PradyX/pocket_llm A privacy-first on-device LLM chat app built with Flutter.	27	Experimental	3	Dart
128	jjhickman/garbanzo-bot Run AI-powered community operations in chat: orchestrate...	27	Experimental	4	TypeScript
129	managelm/linux-windows-agent Lightweight agent for Linux & Windows — natural language server management...	26	Experimental	7	—
130	tsunamayo7/helix-agent Extend Claude Code with local Ollama models — autonomous ReAct agent with...	26	Experimental	5	Python
131	linyeping/GemMate Your sovereign, zero-cost AI study companion—running Gemma 4 entirely on...	25	Experimental	4	Dart
132	kennedyraju55/90-local-llm-projects 🚀 90 Production-Level Projects Built with Local LLM (Gemma 4 + Ollama) -...	25	Experimental	3	Python
133	b-iurea/freelens-ollama-extension Freelens plugin that allow to use local Ollama LLM models with particular...	25	Experimental	4	TypeScript
134	SteveYang05/Mood_Land Moodland — offline-friendly desktop companion with local vision and chat;	25	Experimental	4	TypeScript
135	benmezger/daily-summary Generate daily summaries based on Github PR/Issues using (optionally) Ollama.	25	Experimental	7	Python
136	mahadillahm4di-cyber/mh-gdpr-ai.eu-s-plus Switch GPT ↔ Claude ↔ Llama, never lose context. Your AI brain as 3D stars....	25	Experimental	3	Go
137	BrainDriveAI/BrainDrive A personal AI system that helps you define, set, and reach your goals....	25	Experimental	3	TypeScript
138	zigotica/tilde git bare repo to hold my .bash_profile and .config folder (bash, custom bin...	24	Experimental	4	Shell
139	OzuYatamutsu/project-overengineer An overengineered AI cloud service built for reliability.	23	Experimental	3	TypeScript
140	kubawer21/StealthIt 🛠️ Enhance productivity with StealthIt, a hidden AI utility for screen...	21	Experimental	3	Python
141	94BILLY/NOVA NOVA: A high-performance, bare-metal Win32 C++17 desktop orchestrator....	20	Experimental	7	C++
142	przchojecki/agentic-erdos Agentic workflow for tackling all open Erdos problems at once.	19	Experimental	3	JavaScript
143	ParthSareen/OllamaClaw A minimal take on OpenClaw using Ollama	17	Experimental	4	Go
144	shiinamiyuki/sm120_gemm A Zero-dependency GEMM library for RTX 50 series (SM120). 100% cuBLAS performance.	17	Experimental	4	Cuda
145	Orellius/orellius-cortex-terminal AI-first multi-model terminal. Claude + Ollama + MCP tools in one interface....	12	Experimental	4	TypeScript