interestingLSY/swiftLLM
A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of vLLM).
This project offers a specialized system for experimenting with large language models (LLMs). It takes in LLM weights (like LLaMA variants) and provides high-performance inference outputs, similar to production systems, but within a much smaller, easier-to-modify codebase. It's designed for LLM researchers who need to test novel scheduling or attention mechanisms efficiently.
320 stars. No commits in the last 6 months.
Use this if you are an LLM researcher or academic who needs to rapidly prototype and test new ideas for improving LLM inference performance or explore different architectural components without wading through complex, feature-rich production codebases.
Not ideal if you need a full-featured, production-ready LLM serving solution with broad model and hardware support, or features like quantization, LoRA, and multimodal capabilities out-of-the-box.
Stars
320
Forks
35
Language
Python
License
Apache-2.0
Category
Last pushed
Jun 10, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/interestingLSY/swiftLLM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
alibaba/MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering...
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source,...
tensorzero/tensorzero
TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM...