mburaksayici/smallevals
smallevals — CPU-fast, GPU-blazing fast offline retrieval evaluation for RAG systems with tiny QA models.
This tool helps AI engineers and MLOps practitioners evaluate the retrieval accuracy of their Retrieval Augmented Generation (RAG) systems. It takes your existing vector database connection and embedding model as input. It automatically generates questions from your data chunks, attempts to retrieve the relevant chunks, and then calculates and visualizes the retrieval performance, helping you understand how well your RAG system finds the right information.
Available on PyPI.
Use this if you need a fast, local, and cost-effective way to measure and improve the quality of your RAG system's information retrieval.
Not ideal if you primarily need to evaluate the quality of the generated answers rather than the retrieval of relevant context.
Stars
18
Forks
2
Language
Python
License
—
Category
Last pushed
Dec 04, 2025
Commits (30d)
0
Dependencies
28
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/vector-db/mburaksayici/smallevals"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
danny-avila/rag_api
ID-based RAG FastAPI: Integration with Langchain and PostgreSQL/pgvector
arturoburigo/bfc_script_RAG
RAG for a Domain-Specific-language, using vectorDB and semantic search with LLM to response generation
irfanalidv/ragfallback
ragfallback is a Python library that prevents silent RAG failures — chunk quality, retrieval...
hiatamaworkshop/dcp-rag
Data Cost Protocol encoder for system→AI data injection — converts structured data to...
oguzhankir/omnichunk
Structure-aware text chunking library for code, prose, and markup files. Intelligently splits...