rohit901/VANE-Bench
[NAACL'25] Contains code and documentation for our VANE-Bench paper.
This project offers a specialized benchmark for evaluating how well advanced AI models (Video-LMMs) can spot unusual or inconsistent events in video footage. It takes in video clips, both synthetically generated and from real-world surveillance, and generates question-answer pairs about anomalies. This tool is for AI researchers, particularly those working on computer vision and multi-modal AI, to test and improve their anomaly detection models.
No commits in the last 6 months.
Use this if you are a researcher or developer working with Video-LMMs and need a rigorous way to test their ability to detect subtle, unexpected events or inconsistencies in video content.
Not ideal if you are looking for a plug-and-play solution for real-time anomaly detection in production environments, as this is a research benchmark for model evaluation.
Stars
23
Forks
4
Language
Python
License
MIT
Category
Last pushed
Aug 19, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/rohit901/VANE-Bench"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
stanfordnlp/axbench
Stanford NLP Python library for benchmarking the utility of LLM interpretability methods
aidatatools/ollama-benchmark
LLM Benchmark for Throughput via Ollama (Local LLMs)
LarHope/ollama-benchmark
Ollama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.
qcri/LLMeBench
Benchmarking Large Language Models
THUDM/LongBench
LongBench v2 and LongBench (ACL 25'&24')