punica-ai/punica
Serving multiple LoRA finetuned LLM as one
This project helps developers serve many specialized Large Language Models (LLMs) from a single system efficiently. It takes in requests for different fine-tuned LLMs and provides their specific responses, enabling developers to offer a wide range of AI services without excessive hardware costs. It's for developers building and managing AI applications that use custom, fine-tuned LLMs for various tasks.
1,145 stars. No commits in the last 6 months.
Use this if you are a developer serving multiple custom, fine-tuned Large Language Models and need to maximize throughput and minimize latency with shared resources.
Not ideal if you are only running a single LLM or don't need to serve custom, fine-tuned models.
Stars
1,145
Forks
61
Language
Python
License
Apache-2.0
Category
Last pushed
May 08, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/punica-ai/punica"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
OptimalScale/LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
adithya-s-k/AI-Engineering.academy
Mastering Applied AI, One Concept at a Time
jax-ml/jax-llm-examples
Minimal yet performant LLM examples in pure JAX
young-geng/scalax
A simple library for scaling up JAX programs
riyanshibohra/TuneKit
Upload your data → Get a fine-tuned SLM. Free.