dengls24/LLM-para
Analyze LLM inference: FLOPs, memory, Roofline model. Supports GQA, MoE, MLA, RoPE, SwiGLU. 19 models × 20+ hardware platforms.
This tool helps hardware architects and machine learning engineers understand the performance, energy usage, and cost of running large language models (LLMs) on different hardware. You input details about an LLM (like LLaMA-3) and a hardware platform (like an NVIDIA H100 GPU), and it calculates metrics like FLOPs, memory bottlenecks, throughput, and even carbon footprint. This is for professionals who need to optimize LLM inference deployments.
Use this if you need to quantitatively compare and select the most efficient hardware and LLM architecture configurations for deploying LLMs, considering performance, energy, and cost.
Not ideal if you are looking for a tool to train LLMs, fine-tune models, or evaluate their natural language understanding capabilities.
Stars
10
Forks
1
Language
Python
License
—
Category
Last pushed
Mar 17, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/dengls24/LLM-para"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
alibaba/MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering...
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source,...
tensorzero/tensorzero
TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM...