Local LLM Orchestration Vector Databases

Complete platforms and toolkits for running, managing, and integrating multiple AI components (LLMs, vector databases, agents) entirely on-premises without external APIs. Does NOT include standalone vector databases, individual LLM models, or cloud-dependent AI services.

There are 47 local llm orchestration tools tracked. 1 score above 70 (verified tier). The highest-rated is neuml/txtai at 78/100 with 12,281 stars. 2 of the top 10 are actively maintained.

Get all 47 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=vector-db&subcategory=local-llm-orchestration&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	neuml/txtai 💡 All-in-one AI framework for semantic search, LLM orchestration and...	78	Verified	12,281	Python
2	Mintplex-Labs/anything-llm The all-in-one AI productivity accelerator. On device and privacy first with...	68	Established	56,148	JavaScript
3	DerwenAI/strwythura Strwythura: construct an entity-resolved knowledge graph from structured...	60	Established	214	Python
4	pathwaycom/llm-app Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search...	57	Established	56,145	Jupyter Notebook
5	thebearwithabite/ai-file-organizer Intelligent file organization with computer vision, audio analysis,...	44	Emerging	30	Python
6	Corpus-OS/corpusos Open-source protocol suite standardizing LLM, Vector, Graph, and Embedding...	43	Emerging	144	Python
7	somritdasgupta/aide aiDe is a browser extension that combines a sideKick bar and web UI for...	41	Emerging	5	TypeScript
8	Afnanksalal/co-op Expert guidance across legal, finance, investor relations, and competitive...	39	Emerging	4	TypeScript
9	ArshVermaGit/nexus-ai-life-os NEXUS 🧠 — AI-powered second brain that watches, remembers, and assists in...	38	Emerging	5	Python
10	MicroClub-USTHB/mAIcro mAIcro is a reusable, open-source AI infrastructure designed for communities...	36	Emerging	8	Python
11	kalisam/Amazon_Rose_Forest_01 Amazon Rose Forest is the foundational codebase of FLOSSI0ULLK: a...	33	Emerging	3	Rust
12	codingforentrepreneurs/reflex-gpt Build a Python Reflex App with OpenAI, a Neon Postgres + Vector Database,...	30	Emerging	41	Python
13	localcloud-sh/localcloud Stop paying for AI APIs during development. LocalCloud runs everything...	30	Emerging	46	Go
14	angrysky56/quantum-framework Theoretical basis and prototype concept for an AI native OS as an infinite...	30	Emerging	2	JavaScript
15	PerminovEugene/messy-folder-reorganizer-ai 🤖 AI-powered CLI for file reorganization. Runs fully locally — no data...	29	Experimental	20	Rust
16	piyush-eon/ai-portfolio-nextjs Build an AI Driven Portfolio App with NextJS and Tailwind CSS. We will learn...	29	Experimental	10	JavaScript
17	agentic-dev-io/mao Multi-agent orchestration with MCP — LangChain, DuckDB vector storage,...	27	Experimental	2	Python
18	KingLeoJr/AITools A comprehensive toolkit for building AI-powered applications, covering model...	27	Experimental	12	—
19	gekonge/No-one 🌐 Manage Java WebShells effortlessly with No-one, a modern tool for secure...	26	Experimental	1	TypeScript
20	squaredev-io/squaredev The platform for developers building applications powered by LLMs	26	Experimental	32	TypeScript
21	Rinkatecam/AI.Stack Private AI Infrastructure - Deploy OpenWebUI + Ollama + Qdrant in one...	24	Experimental	1	Shell
22	trustlelab/siteware-backend-v2 Siteware Backend - German Voice AI Agent provider - Deepgram + Twilio +...	24	Experimental	5	TypeScript
23	celersneha/DevZoku DevZoku is a hackathon collaboration platform designed to help developers...	24	Experimental	4	TypeScript
24	dsk-dev-ai/devos-ai 🚀 DevOS AI — Open-source AI operating system for developers. Chat with...	23	Experimental	1	Python
25	Nabarup1/Silvanus Acoustic Nervous System for Forest Protection. An AI-powered multi-agent...	23	Experimental	2	Python
26	jgoy-labs/server-nexe Local AI server with persistent memory, RAG, and multi-backend inference...	22	Experimental	—	Python
27	pramayharsh/omni-mesh A decentralized Multi-Agent AI swarm featuring autonomous memory, security...	22	Experimental	—	Python
28	GrandpaSatan/Yggdrasil Distributed AI service mesh for local LLM orchestration, memory, code...	22	Experimental	—	Rust
29	anubissbe/knowledgehub 🧠 Enterprise AI-Enhanced Development Platform with GraphRAG, LlamaIndex,...	22	Experimental	9	Python
30	ActionpackdHQ/Ignite-AI-SDK Ignite AI SDK - A powerful toolkit for building AI-powered applications with...	22	Experimental	1	TypeScript
31	TensorHub-Technologies/THub-App-Dev-0.1 THub Apps - A Low Code No Code Generative AI Development Platform	21	Experimental	—	TypeScript
32	BrooksErica/voice-biometric-authentication Cloud-native voice biometric authentication system using deep learning...	21	Experimental	—	—
33	irsyadddddds/nexus-ai-toolkit Composable AI agents with vector memory and structured prompt pipelines.	21	Experimental	—	Python
34	24kr/Local_LLM_Assistant LoLA: Privacy-first desktop AI with RAG. Chat offline with your docs using...	21	Experimental	—	JavaScript
35	aicubetechnology/qilbee-mycelial-network Enterprise SaaS platform for AI agent communication inspired by fungal...	21	Experimental	—	Python
36	PinsaraPerera/intellihack_backend This is the backend of viewMo AI cloud platform. AI base graph generation...	19	Experimental	4	Python
37	mrkingsleyobi/trustswarm Real-time blockchain fraud detection using autonomous multi-agent AI swarms....	17	Experimental	—	TypeScript
38	Harsh-SOE/devmatch 🚀 A platform for developers to search for other similar developers by their...	16	Experimental	1	JavaScript
39	Maruch11/LLM-local Documentación técnica y roadmaps para implementación de un LLM local con...	15	Experimental	2	—
40	harshmriduhash/CognitiveHub-AI A unified, AI-powered SaaS platform designed to support engineering teams...	14	Experimental	1	Python
41	gitsudhir/aicli aicli is a Rust-based terminal application that implements a...	13	Experimental	—	Rust
42	hunny345686/FitMetrics FitMetrics AI — Cinematic Resume Intelligence Platform	13	Experimental	—	TypeScript
43	buitoan112233/EleutherIA 🧠 Explore a FAIR-compliant knowledge graph that analyzes ancient debates on...	13	Experimental	—	Shell
44	MassivDash/ai_tools AI Tools is a tester and companion web application designed to help you...	13	Experimental	—	Rust
45	malihanawshin/CloudVoice-agent MCP-based Voice-Ops Agent for Sustainable Cloud Infrastructure Operations.	13	Experimental	—	Python
46	imabutahersiddik/AI-Application-Starter-Kit A starter kit for building AI applications using Neon as a vector database.	11	Experimental	—	JavaScript
47	Jiaxi-Huang/HackerLLM Simple Work	11	Experimental	—	Vue

Comparisons in this category

anything-llm and llm-app (68 vs 57)