cloud-apim/otoroshi-llm-extension
Connect, setup, secure and seamlessly manage LLM models using an Universal/OpenAI compatible API
This extension helps developers and IT operations teams manage and secure access to a wide array of Large Language Models (LLMs) from different providers. It acts as a central gateway, taking requests from your applications and routing them to the best-fit LLM, returning the model's response. It's designed for anyone building applications that rely on multiple AI models, streamlining their deployment and usage.
Use this if you are building applications that interact with various AI models (text, audio, image, video) and need a unified, secure, and resilient way to manage these connections.
Not ideal if you are only using a single LLM provider and don't require advanced features like load balancing, custom quotas, or multi-provider fallbacks.
Stars
15
Forks
5
Language
Scala
License
Apache-2.0
Category
Last pushed
Mar 13, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/cloud-apim/otoroshi-llm-extension"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AlexsJones/llmfit
Hundreds of models & providers. One command to find what runs on your hardware.
victordibia/llmx
An API for Chat Fine-Tuned Large Language Models (llm)
Chen-zexi/vllm-cli
A command-line interface tool for serving LLM using vLLM.
InftyAI/llmaz
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
livehl/aimirror
🚀 200倍速！AI时代的下载神器 | Docker/PyPI/HuggingFace/CRAN 全加速 | 并行分片+智能缓存，让下载飞起来