jpreagan/llmnop
A tool for measuring LLM performance metrics.
This tool helps AI/ML operations engineers and MLOps professionals evaluate the real-world performance of large language models (LLMs) served via API endpoints. You provide details like the API URL, model name, and desired input/output token counts, and it produces detailed metrics on latency (like time to first token) and throughput. This allows you to compare different LLM providers, validate deployments, or optimize serving configurations.
Use this if you need to reliably measure how fast your LLM inference endpoints are responding and generating tokens under various load conditions.
Not ideal if you're a data scientist primarily focused on model accuracy or training, rather than the operational performance of deployed models.
Stars
9
Forks
—
Language
Rust
License
Apache-2.0
Category
Last pushed
Feb 21, 2026
Monthly downloads
38
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/jpreagan/llmnop"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
open-compass/opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral,...
IBM/unitxt
🦄 Unitxt is a Python library for enterprise-grade evaluation of AI performance, offering the...
lean-dojo/LeanDojo
Tool for data extraction and interacting with Lean programmatically.
GoodStartLabs/AI_Diplomacy
Frontier Models playing the board game Diplomacy.
google/litmus
Litmus is a comprehensive LLM testing and evaluation tool designed for GenAI Application...