epfml/llm-optimizer-benchmark
Benchmarking Optimizers for LLM Pretraining
This project offers a standardized way to compare different optimization techniques used in training Large Language Models (LLMs). It takes various optimizer configurations, model sizes, and training durations as input and produces benchmark results showing which optimizer performs best under specific conditions. LLM researchers and practitioners would use this to inform their choice of optimization methods for pretraining LLMs.
Use this if you are pretraining Large Language Models and need to systematically evaluate and select the most effective optimization technique for your specific model size, batch size, or training duration.
Not ideal if you are looking for a tool to train LLMs for immediate application or if your primary focus is fine-tuning an existing LLM for a downstream task.
Stars
56
Forks
4
Language
Python
License
Apache-2.0
Category
Last pushed
Dec 30, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/epfml/llm-optimizer-benchmark"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
stanfordnlp/axbench
Stanford NLP Python library for benchmarking the utility of LLM interpretability methods
aidatatools/ollama-benchmark
LLM Benchmark for Throughput via Ollama (Local LLMs)
LarHope/ollama-benchmark
Ollama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.
qcri/LLMeBench
Benchmarking Large Language Models
THUDM/LongBench
LongBench v2 and LongBench (ACL 25'&24')