gomate-community/rageval

Evaluation tools for Retrieval-augmented Generation (RAG) methods.

/ 100

Emerging

This tool helps evaluate the performance of your Retrieval-Augmented Generation (RAG) systems. It takes the outputs from various stages of your RAG pipeline—like rewritten queries, retrieved documents, and generated answers—and provides comprehensive scores on how well your system is performing across aspects like answer correctness, factual consistency, and document relevance. It is designed for AI/ML engineers or researchers building and refining RAG-based applications.

170 stars. No commits in the last 6 months.

Use this if you are developing or fine-tuning RAG systems and need to systematically measure the quality and effectiveness of different components and the overall output.

Not ideal if you are looking for a simple, out-of-the-box RAG solution without needing to dive deep into component-level performance metrics.

AI-evaluation NLP-benchmarking Generative-AI-testing LLM-performance Information-retrieval-quality

Stale 6m No Package No Dependents

Maintenance 0 / 25

Adoption 10 / 25

Maturity 16 / 25

Community 10 / 25

How are scores calculated?

Stars

170

Forks

Language

Python

License

Apache-2.0

Featured in

You're Shipping AI You Can't Measure

Compare

rageval and open-rag-eval rageval and XRAG rageval and rag-evaluator rageval and RAG-evaluation-harnesses rageval and RAG-Evaluator

Higher-rated alternatives

vectara/open-rag-eval

RAG evaluation without the need for "golden answers"

DocAILab/XRAG

XRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced...

HZYAI/RagScore

⚡️ The "1-Minute RAG Audit" — Generate QA datasets & evaluate RAG systems in Colab, Jupyter, or...

AIAnytime/rag-evaluator

A library for evaluating Retrieval-Augmented Generation (RAG) systems (The traditional ways).

microsoft/benchmark-qed

Automated benchmarking of Retrieval-Augmented Generation (RAG) systems

Explore RAG Tools

All categories Trending RAG directory Insights