CodeEff/ECCO
[EMNLP 2024] Code for the paper "ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?"
This project helps evaluate and improve the efficiency of code generated by large language models, ensuring it runs faster without breaking functionality. It takes either a natural language instruction or existing code with a change history as input, and outputs metrics on the correctness and runtime performance of the generated code. Developers and researchers working with code generation models would use this to benchmark and refine their systems.
No commits in the last 6 months.
Use this if you need to compare how efficiently different large language models generate code, or if you want to test if model-generated code is both correct and fast.
Not ideal if you are looking for a tool to optimize existing human-written code or to debug functional errors in your own applications.
Stars
7
Forks
2
Language
Python
License
—
Category
Last pushed
Oct 03, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ai-coding/CodeEff/ECCO"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
k4black/codebleu
Pip compatible CodeBLEU metric implementation available for linux/macos/win
LiveCodeBench/LiveCodeBench
Official repository for the paper "LiveCodeBench: Holistic and Contamination Free Evaluation of...
EdinburghNLP/code-docstring-corpus
Preprocessed Python functions and docstrings for automated code documentation (code2doc) and...
hendrycks/apps
APPS: Automated Programming Progress Standard (NeurIPS 2021)
solis-team/Hydra
[FSE 2026] Do Not Treat Code as Natural Language: Implications for Repository-Level Code...