datawhalechina/llm-deploy
大模型/LLM推理和部署理论与实践
This project provides practical guidance and theoretical foundations for deploying large language models (LLMs) into production. It helps turn trained LLMs into live services that can handle user requests efficiently. The output is a robust, optimized LLM serving system. This resource is for algorithm engineers and anyone interested in the technical aspects of deploying LLMs.
381 stars. No commits in the last 6 months.
Use this if you are an algorithm engineer or student needing to understand the end-to-end process of taking a large language model from development to a live, performant service.
Not ideal if you are looking for an introduction to training LLMs or their applications, as this focuses specifically on the deployment and inference stages.
Stars
381
Forks
51
Language
—
License
—
Category
Last pushed
Jul 14, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/datawhalechina/llm-deploy"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
PaddlePaddle/FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
mlc-ai/mlc-llm
Universal LLM Deployment Engine with ML Compilation
skyzh/tiny-llm
A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny...
ServerlessLLM/ServerlessLLM
Serverless LLM Serving for Everyone.
AXERA-TECH/ax-llm
Explore LLM model deployment based on AXera's AI chips