xlang-ai/DS-1000
[ICML 2023] Data and code release for the paper "DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation".
This project provides a standardized collection of data science problems, complete with descriptions, expected inputs, and correctness checks. It helps evaluate how well AI models can generate code for common data science tasks using libraries like Pandas, NumPy, and Matplotlib. Data science practitioners and AI researchers who develop or benchmark code generation models would find this valuable for assessing model performance.
267 stars. No commits in the last 6 months.
Use this if you are developing or evaluating AI models that generate Python code for data science tasks and need a consistent, reliable benchmark.
Not ideal if you are a data scientist looking for a tool to directly assist with your daily data analysis or machine learning tasks.
Stars
267
Forks
30
Language
Python
License
CC-BY-SA-4.0
Category
Last pushed
Oct 30, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/generative-ai/xlang-ai/DS-1000"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.