LLM Orchestration Platforms Embedding Tools

Tools for unifying access to multiple LLM providers, managing model selection, routing requests, and coordinating AI agent execution across different backends. Does NOT include domain-specific applications, single-provider SDKs, or deployment infrastructure (unless orchestration is the primary purpose).

There are 74 llm orchestration platforms tools tracked. 4 score above 50 (established tier). The highest-rated is lfnovo/esperanto at 63/100 with 157 stars.

Get all 74 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=embeddings&subcategory=llm-orchestration-platforms&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	lfnovo/esperanto A unified interface for various AI model providers	63	Established	157	Python
2	apocas/restai RESTai is an AIaaS (AI as a Service) open-source platform. Built on top of...	60	Established	480	Python
3	baidubce/bce-qianfan-sdk Provide best practices for LMOps, as well as elegant and convenient access...	52	Established	383	Jupyter Notebook
4	sbhjt-gr/InferrLM On-device AI for iOS & Android	52	Established	66	TypeScript
5	ProviderProtocol/ai 0-DEP AI DX SDK	47	Emerging	3	TypeScript
6	Muvon/octolib The lib to power AI tools.	46	Emerging	2	Rust
7	spring-petclinic/spring-petclinic-langchain4j Spring Petclinic application with a chatbot powered by OpenAI's Generative...	45	Emerging	16	Java
8	solygambas/python-openai-projects 13 projects using ChatGPT API, Whisper, Embeddings, and DALL-E with Python.	44	Emerging	51	Jupyter Notebook
9	itayzit/openai-async A light-weight, asynchronous client for OpenAI API - text completion, image...	44	Emerging	56	Python
10	IBM/watsonx-ai-java-sdk The watsonx.ai Java SDK is an open-source library that simplifies the...	43	Emerging	5	Java
11	maragudk/gai Go Artificial Intelligence (GAI) helps you work with foundational models,...	43	Emerging	41	Go
12	rrg92/powershai Powershell + AI	42	Emerging	61	PowerShell
13	jchristn/SharpAI SharpAI is an embeddable embeddings, completions, and model management...	42	Emerging	6	C#
14	SANSA-Stack/Archived-SANSA-ML SANSA Machine Learning Layer	41	Emerging	39	Scala
15	UBOS-tech/node-red-contrib-openai-ubos A Node-RED node that interacts with OpenAI machine learning models to...	41	Emerging	12	JavaScript
16	lpalbou/AbstractCore A unified Python library for interaction with multiple Large Language Model...	38	Emerging	5	Python
17	ramanujammv1988/edge-veda On-device AI SDK for Flutter — LLM inference, vision, STT, TTS, image...	38	Emerging	74	Dart
18	netresearch/t3x-nr-llm The shared AI foundation for TYPO3 — one LLM setup for every extension on your site	36	Emerging	1	PHP
19	eliben/gemini-cli Access Gemini LLMs from the command-line	36	Emerging	148	Go
20	connerohnesorge/semanticrouter-go Fast & less costly AI decision making and intelligent processing of multi-modal data.	35	Emerging	11	Go
21	Prismadic/magnet the small distributed language model toolkit; fine-tune state-of-the-art...	34	Emerging	32	Python
22	kingabzpro/easy-local-ai Run Ollama (Llama3.2), Langflow, Postgres, and qdrant server using Docker Compose.	33	Emerging	6	—
23	proj-airi/webai-examples 🧠 Web AI / LLM in browser / Whisper in browser / WebGPU inference Examples	32	Emerging	28	Vue
24	wizenheimer/tinylm Browser based ML Inference \| OpenAI compliant \| Run models like DeepSeek,...	31	Emerging	52	TypeScript
25	priyanshua44/no-llm no-llm is a lightweight library designed to simplify the integration of...	31	Emerging	1	Python
26	pinecone-io/gemini-cli-extension The official Pinecone Gemini CLI extension repo.	30	Emerging	18	Python
27	skorotkiewicz/llmnet The Offline Internet.	29	Experimental	65	TypeScript
28	cheahjs/gemini-to-openai-proxy Call Gemini (https://ai.google.dev) embedding models with OpenAI-compatible endpoints	29	Experimental	3	Go
29	nshkrdotcom/ollixir Ollixir provides a first-class Elixir client with feature parity to the...	26	Experimental	3	Elixir
30	symfony/ai-open-ai-platform OpenAI platform bridge for Symfony AI	26	Experimental	3	PHP
31	bennyschmidt/next-token-prediction Next-token prediction in JavaScript — build fast language and diffusion models.	26	Experimental	143	JavaScript
32	burgerkhan6227/tokenWise-Optimizer 🎯 Optimize LLM token usage by 70-90% with smart context ranking, reducing...	25	Experimental	2	Python
33	Netsleek/the-selection-layer Defines the Selection Layer — the decision system through which AI models...	23	Experimental	1	—
34	codenameakshay/ai_sdk_dart Dart/Flutter port of Vercel AI SDK v6 — provider-agnostic text generation,...	22	Experimental	—	Dart
35	Naseem77/tokenWise-Optimizer Smart Context Optimization for LLMs - Reduce tokens by 66%, save 40% on API...	22	Experimental	4	Python
36	PabloSanchi/IBM-WatsonxAI-Spring-AI-Example Example of IBM watsonx.ai with Spring AI	22	Experimental	1	Java
37	liliang-cn/agent-go AI Agent SDK designed for Go developers	21	Experimental	6	Go
38	leafspec/spec Framework-agnostic specification for AI-native applications. LEAF (Listen,...	21	Experimental	—	—
39	Aman00000007/lexilux 🚀 Simplify API calls with Lexilux, a unified LLM client that lets you access...	21	Experimental	—	Python
40	bernard777/tool-selector-cascade Cascading tool selector for LLM agents — narrows 1000+ tools to the best...	21	Experimental	—	Python
41	bgokden/llama-embedding High-performance, thread-safe embedding library using llama.cpp.	21	Experimental	—	C++
42	kamil5b/go-nl2query-lib A Go library for converting natural language queries into database queries...	21	Experimental	—	Go
43	m1thrandir225/galore-services The main services for the Galore mobile app.	21	Experimental	—	Go
44	bgokden/llama-reranker High-performance, thread-safe reranking library using llama.cpp	21	Experimental	—	C++
45	Dragon-Born/go-llm Developer-friendly Go SDK for LLM apps: OpenAI, Anthropic, Gemini, Ollama,...	21	Experimental	2	Go
46	languageseed/valet-gateway AI Inference Gateway - orchestrates Ollama, vLLM, cloud providers, and...	21	Experimental	2	Python
47	sahil-makandar/beyond-chatgpt-ai-systems Guest lecture presentation explaining modern AI architecture: LLMs, RAG...	21	Experimental	—	—
48	douglasmitsue/ai-systems-engineering-portfolio Designing and building scalable AI systems - from deep neural networks to...	21	Experimental	—	—
49	superdupers1/ai-thought-visual 🎨 Transform language, voice, and images into structured AI concepts,...	19	Experimental	1	Lex
50	Ammar-Alnagar/Axion Axion is a high-performance LLM serving platform built with Rust that...	18	Experimental	1	Rust
51	LLM-Gateway-ORG/llm-gateway-core A Python package to access different LLMs, embeddings, vector stores etc.	18	Experimental	2	Python
52	FANMixco/openai-outsystems-wrapper A simple wrapper for the OpenAI APIs for OutSystems	17	Experimental	1	—
53	linzeyang/minimax-python-client An (unofficial) python native client for easy interaction with MiniMax Open Platform	17	Experimental	—	Python
54	inkybubble/agents-01-foundation An LLM enhanced with tools and retrieval (no memory for now)	17	Experimental	—	Python
55	stiebo/spring-ai-samples Spring AI, chat client, vector store, RAG, multimodality samples	14	Experimental	1	Java
56	iChetanRaval/Steganography-Project A professional-grade steganography platform that enables secure data hiding...	14	Experimental	3	Python
57	jtgsystems/Ollama-Menu-main 🦙 Ollama menu system - Streamlined AI model management	14	Experimental	1	Batchfile
58	sanketrana2598/ai-terminology 📘 Discover key AI terms and their alternatives in this concise guide to...	13	Experimental	—	Jupyter Notebook
59	orchestra-mcp/pack-ai AI and LLM integration skills with RAG, embeddings, and vector search	13	Experimental	—	—
60	xp-forge/openai OpenAI APIs for XP Framework	13	Experimental	—	PHP
61	dogunkim/llmnet 🔍 Transform your local LLMs into a private, high-speed search engine,...	13	Experimental	—	TypeScript
62	vs4vijay/LLM-Ecosystem Code for Embeddings, VectorStore, SemanticSearch, and RAG using Azure OpenAI	13	Experimental	—	Python
63	dotcommander/syn Fast CLI for the Synthetic.new AI API — chat, search, vision, embeddings....	13	Experimental	2	Go
64	redoh/ollama-recipes 🦙 Local LLM recipes with Ollama — fine-tuning, RAG, embeddings, multi-model pipelines	13	Experimental	—	—
65	zzarif/AI-Detector Detect AI generated coding answers	13	Experimental	—	JavaScript
66	ArchitJ6/What-Beats-AI What Beats AI is an interactive word-challenge game that leverages...	13	Experimental	—	Python
67	pipewrk/llm-core Lightweight, composable TypeScript library for semantic chunking, workflow...	13	Experimental	—	TypeScript
68	onfiiva/ai-assistant-api LLM API with RAG, Agent, Embeddings and open source models	13	Experimental	—	Python
69	johnamit/semantic-context-tokens A hybrid tokenization framework that combines coarse semantic context tokens...	13	Experimental	—	—
70	Al-Aswadd/Augment-BYOK-Proxy 🚀 Transform your Augment experience with the BYOK proxy for seamless LLM...	13	Experimental	—	Rust
71	stephengroe/little-language-model 🤖 A tiny GPT-style language model built from scratch. Built to explore ML...	12	Experimental	2	TypeScript
72	miladhub/chat-ai Example of using a conversational AI with embeddings with Java	11	Experimental	—	Java
73	sonufrienko/ai-engineering OpenAI, LLM, LangChain, LlamaIndex, Vector Search	11	Experimental	—	Jupyter Notebook
74	SkywardAI/lupinIII Restful API aggregators in Rust which focus on high performance, Rust AI...	11	Experimental	—	Rust