RUC-NLPIR/OmniEval
Open source code of the paper: "OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain"
This project helps financial domain experts and AI engineers automatically evaluate the performance of Retrieval Augmented Generation (RAG) systems. It takes a knowledge corpus and generates evaluation data samples, then allows you to assess your RAG models' accuracy, completeness, utilization, numerical accuracy, and hallucination in financial contexts. Financial analysts, data scientists specializing in finance, or AI researchers building financial RAG applications would use this.
No commits in the last 6 months.
Use this if you need an automated, comprehensive way to benchmark your RAG models' effectiveness using real-world financial data.
Not ideal if you are evaluating RAG systems outside of the financial domain or require purely human-in-the-loop evaluation methods.
Stars
82
Forks
7
Language
Python
License
MIT
Category
Last pushed
Dec 20, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/RUC-NLPIR/OmniEval"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
vectara/open-rag-eval
RAG evaluation without the need for "golden answers"
DocAILab/XRAG
XRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced...
HZYAI/RagScore
⚡️ The "1-Minute RAG Audit" — Generate QA datasets & evaluate RAG systems in Colab, Jupyter, or...
AIAnytime/rag-evaluator
A library for evaluating Retrieval-Augmented Generation (RAG) systems (The traditional ways).
microsoft/benchmark-qed
Automated benchmarking of Retrieval-Augmented Generation (RAG) systems