xllm and ZhiLight
Both are high-performance LLM inference engines optimized for acceleration, making them competitors in the same category.
About xllm
jd-opensource/xllm
A high-performance inference engine for LLMs, optimized for diverse AI accelerators.
This project helps businesses and organizations deploy large language models (LLMs) like DeepSeek-V3.1 or Qwen2/3, especially on Chinese AI accelerators. It takes these pre-trained models and makes them run much faster and more cost-effectively, generating text responses for applications like intelligent customer service, risk control, or ad recommendations. The end-users are AI solution architects, MLOps engineers, and IT infrastructure managers responsible for deploying and managing AI applications.
About ZhiLight
zhihu/ZhiLight
A highly optimized LLM inference acceleration engine for Llama and its variants.
ZhiLight is a specialized engine designed to speed up the process of generating text from large language models (LLMs) like Llama and its variants. It takes your trained LLM and, by optimizing how the model runs on NVIDIA GPUs, delivers faster responses and more outputs per second. This tool is for AI engineers or machine learning operations specialists who deploy and manage LLMs in production.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work