hendrycks/apps
APPS: Automated Programming Progress Standard (NeurIPS 2021)
This project provides a comprehensive dataset and tools for evaluating the ability of large language models to solve coding challenges. It takes programming problems as input and measures how well a model can generate correct, executable code. Researchers and developers working on AI that writes code would use this to benchmark and improve their models.
520 stars. No commits in the last 6 months.
Use this if you are a researcher or developer focused on training and evaluating AI models that can generate solutions to programming problems.
Not ideal if you are looking for an application that helps humans write code, or a tool for automated software testing.
Stars
520
Forks
70
Language
Python
License
MIT
Category
Last pushed
Jun 19, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ai-coding/hendrycks/apps"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
k4black/codebleu
Pip compatible CodeBLEU metric implementation available for linux/macos/win
LiveCodeBench/LiveCodeBench
Official repository for the paper "LiveCodeBench: Holistic and Contamination Free Evaluation of...
EdinburghNLP/code-docstring-corpus
Preprocessed Python functions and docstrings for automated code documentation (code2doc) and...
solis-team/Hydra
[FSE 2026] Do Not Treat Code as Natural Language: Implications for Repository-Level Code...
alxschwrz/codex_py2cpp
Converts python code into c++ by using OpenAI CODEX.