EvilFreelancer/benchmarking-llms
Comprehensive benchmarks and evaluations of Large Language Models (LLMs) with a focus on hardware usage, generation speed, and memory requirements.
This benchmark provides a clear comparison of various large language models (LLMs) to help you choose the right one for your needs. It details how different LLMs perform in terms of hardware usage like VRAM and initial RAM, along with their generation speed and output length. AI engineers and researchers can use this information to optimize model deployment and resource allocation.
No commits in the last 6 months.
Use this if you need to select an LLM for deployment and are concerned about its real-world performance metrics such as generation speed and memory footprint on specific hardware.
Not ideal if you are looking for qualitative evaluations of LLM output, such as creativity or factual accuracy, rather than technical performance metrics.
Stars
12
Forks
1
Language
Python
License
—
Category
Last pushed
Aug 31, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/EvilFreelancer/benchmarking-llms"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
TsinghuaC3I/MARTI
A Framework for LLM-based Multi-Agent Reinforced Training and Inference
zjunlp/KnowLM
An Open-sourced Knowledgable Large Language Model Framework.
cli99/llm-analysis
Latency and Memory Analysis of Transformer Models for Training and Inference
tanyuqian/redco
NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to...
stanleylsx/llms_tool
一个基于HuggingFace开发的大语言模型训练、测试工具。支持各模型的webui、终端预测，低参数量及全参数模型训练(预训练、SFT、RM、PPO、DPO)和融合、量化。