predibase/lorax
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
This tool helps machine learning engineers and MLOps teams efficiently manage and deploy many specialized AI models (called 'adapters') that run on top of larger foundational models. It takes your fine-tuned LoRA adapters and a base Large Language Model, and provides a scalable inference server. The output is fast, cost-effective text generation from any of your specialized models.
3,735 stars. No commits in the last 6 months.
Use this if you need to serve hundreds or thousands of fine-tuned large language models without incurring massive GPU costs or compromising on response speed.
Not ideal if you only need to run a single large language model or don't have GPU infrastructure available.
Stars
3,735
Forks
309
Language
Python
License
Apache-2.0
Category
Last pushed
May 21, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/predibase/lorax"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
unslothai/unsloth
Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, DeepSeek, Qwen, Llama,...
huggingface/peft
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
modelscope/ms-swift
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.5, DeepSeek-R1, GLM-5,...
oumi-ai/oumi
Easily fine-tune, evaluate and deploy gpt-oss, Qwen3, DeepSeek-R1, or any open source LLM / VLM!
linkedin/Liger-Kernel
Efficient Triton Kernels for LLM Training