PatrickKoss/genai-gateway
A llm gateway for centralized rate limiting and metrics for OpenAI compatible model endpoints.
This gateway helps developers manage interactions with large language models like OpenAI, Anthropic, or self-hosted LLMs. It takes developer requests for LLM interactions and outputs responses while handling important backend tasks. Developers building applications that use LLMs will find this useful for managing costs and performance.
No commits in the last 6 months.
Use this if you are developing an application that uses large language models and need a centralized way to control usage, track costs, and connect to multiple LLM providers.
Not ideal if you are an end-user simply interacting with an LLM directly and don't need to manage an application's access to them.
Stars
7
Forks
—
Language
Rust
License
—
Category
Last pushed
Sep 20, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/PatrickKoss/genai-gateway"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
BerriAI/litellm
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with...
vava-nessa/free-coding-models
Find, benchmark and install in CLI 158 FREE coding LLM models across 20 providers in real time
envoyproxy/ai-gateway
Manages Unified Access to Generative AI Services built on Envoy Gateway
theopenco/llmgateway
Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
Portkey-AI/gateway
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with...