Chemistry LLM Benchmarks LLM Tools

Tools, datasets, and benchmarks for evaluating and fine-tuning large language models on chemistry and molecular property prediction tasks. Does NOT include general scientific LLM frameworks, materials science benchmarks, or chemistry software without LLM components.

There are 17 chemistry llm benchmarks tools tracked. 2 score above 50 (established tier). The highest-rated is theochem/ModelHamiltonian at 54/100 with 55 stars.

Get all 17 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=llm-tools&subcategory=chemistry-llm-benchmarks&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	theochem/ModelHamiltonian Generate 1- and 2-electron integrals so that molecular quantum chemistry...	54	Established	55	Python
2	lamalab-org/chembench How good are LLMs at chemistry?	50	Established	134	Python
3	pnnl/cactus LLM Agent that leverages cheminformatics tools to provide informed responses.	44	Emerging	48	Jupyter Notebook
4	jan-janssen/LangSim Application of Large Language Models (LLM) for computational materials...	43	Emerging	84	Jupyter Notebook
5	MasterAI-EAM/Darwin An open-source project dedicated to build foundational large language model...	41	Emerging	247	Jupyter Notebook
6	andresilvapimentel/AI4Chem AI4Chem is a code to test the ability of large language models (ChatGPT) to...	40	Emerging	23	Jupyter Notebook
7	lamalab-org/chemlift Language-interfaced fine-tuning for chemistry	38	Emerging	45	Jupyter Notebook
8	lamalab-org/macbench Probing the limitations of multimodal language models for chemistry and...	36	Emerging	23	Python
9	jschrier/SynthGPT Code and Data for "Large Language Models for Inorganic Synthesis Prediction"	34	Emerging	33	Python
10	lamalab-org/chem-bench-app Frontend for evaluating humans on chemistry questions	33	Emerging	11	TypeScript
11	google/task-oriented-queries Task-oriented queries (e.g., one-shot queries to play videos, order food, or...	32	Emerging	5	—
12	chemkg/c3p LLM-generated CHEBI classifiers	30	Emerging	13	Python
13	ChemFoundationModels/ChemLLMBench Official Code for What can Large Language Models do in chemistry? A...	26	Experimental	170	Jupyter Notebook
14	renjieli08/QuantumChem-200K QuantumChem-200K: A Large-Scale Open Organic Molecular Dataset for...	25	Experimental	2	Jupyter Notebook
15	ai4cat/AI4C-LitMiner Developed for AI-driven catalyst discovery, integrating LLM-based knowledge...	22	Experimental	—	Python
16	jschrier/KRICT_hackathon_phosphors KRICT ChemDX Hackathon project: Inorganic Phosphors	19	Experimental	4	Mathematica
17	apekshyasharma/AAII_Intelligence_Idex_Analysis A data-driven benchmarking analysis of leading Artificial Intelligence...	13	Experimental	—	Jupyter Notebook

Comparisons in this category

chembench and ChemLLMBench (50 vs 26) chembench and macbench (50 vs 36)