rtp-llm and xllm
About rtp-llm
alibaba/rtp-llm
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
This is a high-performance engine for deploying large language models (LLMs) in real-world applications. It takes your trained LLM, potentially with multimodal inputs like images and text, and efficiently generates responses for a large number of users. It is designed for engineers and AI product managers responsible for running LLM-powered services like AI assistants or smart search features at scale.
About xllm
jd-opensource/xllm
A high-performance inference engine for LLMs, optimized for diverse AI accelerators.
This project helps businesses and organizations deploy large language models (LLMs) like DeepSeek-V3.1 or Qwen2/3, especially on Chinese AI accelerators. It takes these pre-trained models and makes them run much faster and more cost-effectively, generating text responses for applications like intelligent customer service, risk control, or ad recommendations. The end-users are AI solution architects, MLOps engineers, and IT infrastructure managers responsible for deploying and managing AI applications.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work