actiontech/sql-llm-benchmark
SCALE: SQL Capability Leaderboard for LLMs
This project helps data professionals and database administrators assess and compare how well different Large Language Models (LLMs) handle SQL tasks. It takes a list of LLMs and SQL-related test cases, then produces a detailed report and a public leaderboard showing each LLM's proficiency in understanding SQL, converting SQL dialects, and optimizing SQL queries. The end-user is typically someone who needs to choose or fine-tune an LLM for SQL-related applications.
Use this if you need to objectively benchmark and compare various LLMs based on their SQL capabilities to select the best model for database interactions or query generation.
Not ideal if you are looking for a tool to generate SQL queries directly from natural language or to perform routine database administration tasks.
Stars
23
Forks
3
Language
TypeScript
License
MIT
Category
Last pushed
Mar 09, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/actiontech/sql-llm-benchmark"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
sierra-research/tau2-bench
τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment
xlang-ai/OSWorld
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
bigcode-project/bigcodebench
[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI
THUDM/AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
scicode-bench/SciCode
A benchmark that challenges language models to code solutions for scientific problems