cloudmercato/ollama-benchmark
Handy tool to measure the performance and efficiency of LLMs workloads.
This tool helps AI engineers and researchers assess how well their Ollama-hosted large language models (LLMs) are performing. It takes various LLM models and test parameters as input and outputs detailed performance metrics like response speed, embedding generation time, and even the quality of answers. You can use it to compare different models or optimize a single model's setup for specific tasks.
No commits in the last 6 months.
Use this if you need to systematically measure and compare the speed, efficiency, and quality of different LLM configurations running on Ollama.
Not ideal if you are looking for a tool to deploy or manage your LLMs, or if you need to benchmark models hosted on platforms other than Ollama.
Stars
76
Forks
8
Language
Python
License
MIT
Category
Last pushed
Apr 26, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/cloudmercato/ollama-benchmark"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Higher-rated alternatives
stanfordnlp/axbench
Stanford NLP Python library for benchmarking the utility of LLM interpretability methods
aidatatools/ollama-benchmark
LLM Benchmark for Throughput via Ollama (Local LLMs)
LarHope/ollama-benchmark
Ollama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.
qcri/LLMeBench
Benchmarking Large Language Models
THUDM/LongBench
LongBench v2 and LongBench (ACL 25'&24')