miko-ai-org/llmbatching
An openAI / LLM API wrapper that simplifies the batching API to reduce costs by 50%
This project helps software developers reduce costs when using large language model (LLM) APIs like OpenAI for high-volume, non-time-sensitive tasks. It takes individual API requests from your application, batches them automatically, and sends them to the LLM provider, abstracting away all the complexity. The result is significantly lower API costs for workflows like bulk data classification or periodic evaluations.
No commits in the last 6 months.
Use this if you are a developer integrating LLMs into an application and need to process large volumes of requests where real-time responses are not critical, such as for backend data processing or offline analytics.
Not ideal if your application requires immediate, real-time responses from LLM APIs, as batch processing introduces a delay that can range from minutes to hours.
Stars
19
Forks
2
Language
TypeScript
License
MIT
Category
Last pushed
May 26, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/miko-ai-org/llmbatching"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
microsoft/multilspy
multilspy is a lsp client library in Python intended to be used to build applications around...
mlc-ai/xgrammar
Fast, Flexible and Portable Structured Generation
vicentereig/dspy.rb
The Ruby framework for programming—rather than prompting—language models.
feenkcom/gt4llm
A GT package for working with LLMs
Evref-BL/Pharo-LLMAPI
Use LLM API from Pharo