syy12335/rag-eval-scaffold
Lightweight, decoupled RAG evaluation scaffold (dataset → vector store → RAG workflow → evaluation) with a Qwen-based demo.
This project helps RAG and LLM engineers quickly build, test, and compare different AI chat systems. You provide a dataset of questions and answers, and it automatically builds a knowledge base, simulates a chat workflow, and then evaluates how well the system performs. This is ideal for professionals developing or improving AI assistants that rely on external information.
Use this if you need to rapidly experiment with and compare different RAG configurations or LLM models, from data ingestion to final evaluation.
Not ideal if you're looking for a low-code platform or a ready-to-deploy RAG application without any coding or configuration.
Stars
17
Forks
4
Language
Python
License
MIT
Category
Last pushed
Dec 11, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/syy12335/rag-eval-scaffold"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
vectara/open-rag-eval
RAG evaluation without the need for "golden answers"
DocAILab/XRAG
XRAG: eXamining the Core - Benchmarking Foundational Component Modules in Advanced...
HZYAI/RagScore
⚡️ The "1-Minute RAG Audit" — Generate QA datasets & evaluate RAG systems in Colab, Jupyter, or...
AIAnytime/rag-evaluator
A library for evaluating Retrieval-Augmented Generation (RAG) systems (The traditional ways).
microsoft/benchmark-qed
Automated benchmarking of Retrieval-Augmented Generation (RAG) systems