Vector DB Benchmarking Vector Databases

Performance evaluation frameworks and tools for comparing vector databases across metrics like latency, throughput, recall, and cost. Includes benchmarking suites, tuning optimization systems, and comparative analysis frameworks. Does NOT include individual vector database implementations, retrieval pipeline frameworks, or production RAG applications.

There are 29 vector db benchmarking tools tracked. 1 score above 70 (verified tier). The highest-rated is lancedb/lancedb at 81/100 with 9,425 stars. 2 of the top 10 are actively maintained.

Get all 29 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=vector-db&subcategory=vector-db-benchmarking&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	lancedb/lancedb Developer-friendly OSS embedded retrieval library for multimodal AI. Search...	81	Verified	9,425	HTML
2	zilliztech/VectorDBBench Benchmark for vector databases.	68	Established	1,038	Python
3	qdrant/vector-db-benchmark Framework for benchmarking vector search engines	61	Established	353	Python
4	prrao87/lancedb-study Comparing LanceDB and Elasticsearch for full-text search and vector search...	49	Emerging	29	Python
5	vector-index-bench/vibe Vector Index Benchmark for Embeddings (VIBE) is an extensible benchmark for...	46	Emerging	36	Python
6	myscale/vector-db-benchmark Framework for benchmarking fully-managed vector databases	44	Emerging	80	Python
7	avnlp/vectordb Production-ready Haystack/LangChain pipelines for Hybrid & Parent-Child...	41	Emerging	4	Python
8	tiannuo-yang/VDTuner [ICDE 2024] VDTuner - Automated Performance Tuning for Vector Data...	40	Emerging	35	Python
9	lancedb/docs Documentation for LanceDB, the Multimodal Lakehouse for AI	38	Emerging	6	Python
10	microsoft/VBench An Approximate Vector-Analytics Benchmark for Relational Databases	31	Emerging	2	Python
11	scientist-labs/lancelot Ruby bindings for the Lance columnar data format. Built on the Lance Rust...	29	Experimental	7	Ruby
12	lennylxx/lancedb-csharp LanceDB C# SDK	27	Experimental	1	C#
13	M4iKZ/Vector-Arena A comprehensive, multiprocessing-isolated benchmark for evaluating vector...	23	Experimental	1	Python
14	Praxis-Collective/tdr-vector-decay Native temporal decay scoring for vector database retrieval — no...	22	Experimental	—	Python
15	gordonmurray/metabare.com A lightweight digital asset manager (DAM) focused on simplicity, fast...	20	Experimental	4	Python
16	Devparihar5/chromdb-vs-pgvector-benchmark This repository contains a comprehensive benchmark suite for comparing...	18	Experimental	1	Python
17	Eurphus/Vector-DB-Testing Vector Database testing for semantic search capabilities for use in LLM applications	18	Experimental	2	Python
18	ktsouvalis/mscacs-thesis Official implementation for the MSc Thesis: "Performance Evaluation of...	17	Experimental	—	Python
19	haja-k/tidb-vector-llm-testbed Experimental framework for evaluating TiDB’s vector search capabilities with...	17	Experimental	—	Python
20	shift-ai007/vector-database-guide Vector database comparison for production RAG — Pinecone vs pgvector vs...	14	Experimental	—	—
21	tpancholi/vector-db-benchmark A python repository to benchmark different vector db based on search...	13	Experimental	—	Python
22	bijay-odyssey/vectorDB A Qdrant Vector Database demo for semantic search and RAG, using Sentence...	13	Experimental	—	Python
23	offlabel-scaffolds/scaffold-vector-db-app Vector database application with Qdrant/Weaviate	13	Experimental	—	Python
24	aimanmuzaffar95/vector-databases-benchmarks Benchmark and compare vector databases (pgvector, Qdrant, Milvus, Weaviate,...	13	Experimental	—	Python
25	PeterShin23/db-benchmark Comparing popular Vector DB options used for RAG search	13	Experimental	—	Python
26	euiyounghwang/Vector_DB_with_LLM Vector_DB_with_LLM	12	Experimental	2	Jupyter Notebook
27	Ashish-Soni08/DKE-Thesis-OVGU Master thesis research and implementation for the Data and Knowledge...	11	Experimental	—	—
28	sahilfaizal01/Large-Scale-Vector-DB This project focuses on developing a scalable vector database system...	11	Experimental	—	Python
29	Ezzaldin97/News-Vector-Store Building a News Vector Store by getting latest news from API and load...	10	Experimental	2	Jupyter Notebook

Comparisons in this category

lancedb and lancedb-study (81 vs 49) lancedb and lancelot (81 vs 29) lancedb and docs (81 vs 38) lancedb and lancedb-csharp (81 vs 27) VectorDBBench and vector-db-benchmark (68 vs 61) VectorDBBench and vector-db-benchmark (68 vs 44) VectorDBBench and VBench (68 vs 31) VectorDBBench and Vector-Arena (68 vs 23) vector-db-benchmark and VBench (61 vs 31) vector-db-benchmark and Vector-Arena (61 vs 23)