yuhui-zh15/AutoConverter
Official implementation of "Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation" (CVPR 2025)
This tool helps researchers and evaluators of Vision Language Models (VLMs) by transforming existing open-ended visual question-answering (VQA) datasets into a multiple-choice format. It takes images and their original open-ended questions as input, and outputs challenging multiple-choice questions, along with correct and plausible incorrect answers. This makes VLM evaluation more objective and efficient for AI researchers and machine learning engineers.
No commits in the last 6 months.
Use this if you need to create standardized, challenging multiple-choice questions from open-ended VQA datasets to rigorously evaluate different Vision Language Models.
Not ideal if you are looking to generate brand new visual questions or if your primary goal is not VLM evaluation.
Stars
40
Forks
2
Language
Python
License
—
Category
Last pushed
May 26, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/yuhui-zh15/AutoConverter"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Pinafore/qb
QANTA Quiz Bowl AI
KristiyanVachev/Question-Generation
Generating multiple choice questions from text using Machine Learning.
wuba/qa_match
A simple effective ToolKit for short text matching
PolyAI-LDN/conversational-datasets
Large datasets for conversational AI
mcQA-suite/mcQA
🔮 Answering multiple choice questions with Language Models.