Evaluation Frameworks Metrics RAG Tools
There are 5 evaluation frameworks metrics tools tracked. The highest-rated is amazon-science/auto-rag-eval at 41/100 with 86 stars.
Get all 5 projects as JSON
curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=rag&subcategory=evaluation-frameworks-metrics&limit=20"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
| # | Tool | Score | Tier |
|---|---|---|---|
| 1 |
amazon-science/auto-rag-eval
Code repo for the ICML 2024 paper "Automated Evaluation of... |
|
Emerging |
| 2 |
ibm-self-serve-assets/JudgeIt-LLM-as-a-Judge
Automation Framework using LLM-as-a-judge to evaluate of Agentic AI, RAG,... |
|
Emerging |
| 3 |
explore-de/rage4j
Evaluate your LLM based Java Apps |
|
Emerging |
| 4 |
mit-ll-ai-technology/llm-sandbox
Large language model evaluation framework for logic and open-ended Q&A with... |
|
Emerging |
| 5 |
nl4opt/ORQA
[AAAI 2025] ORQA is a new QA benchmark designed to assess the reasoning... |
|
Experimental |