OSU-NLP-Group/llm-planning-eval
[ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"
This project offers a framework and data to evaluate how well different large language model (LLM) planning strategies perform, specifically for tasks like converting natural language questions into database queries (text-to-SQL) or solving math word problems. It takes in structured datasets of natural language tasks and outputs performance metrics, helping researchers and developers understand when tree search methods enhance LLM performance. Data scientists or AI researchers working with LLMs for complex reasoning tasks would use this.
No commits in the last 6 months.
Use this if you are a researcher or developer who needs to rigorously test and compare various LLM planning techniques for tasks involving database interaction or mathematical reasoning.
Not ideal if you are looking for a ready-to-use LLM application or a tool for general natural language processing tasks outside of planning evaluation.
Stars
54
Forks
3
Language
Python
License
—
Category
Last pushed
Feb 23, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/OSU-NLP-Group/llm-planning-eval"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
open-thought/reasoning-gym
[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards
Hmbown/Hegelion
Dialectical reasoning architecture for LLMs (Thesis → Antithesis → Synthesis)
LLM360/Reasoning360
A repo for open research on building large reasoning models
TsinghuaC3I/Awesome-RL-for-LRMs
A Survey of Reinforcement Learning for Large Reasoning Models
bowang-lab/BioReason
BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model | NeurIPS '25