GT-Vision-Lab/GuessWhich

Evaluating Visual Conversational Agents via Cooperative Human-AI Games

30
/ 100
Emerging

This project helps researchers and AI developers evaluate how well visual AI agents perform when collaborating with humans. It sets up a 'GuessWhich' game where a human asks an AI questions about a hidden image to identify it from a set of choices. The output is a measure of human-AI team performance, specifically how many guesses it takes the human to identify the image.

No commits in the last 6 months.

Use this if you are an AI researcher or developer wanting to test how well your visual conversational AI model integrates and performs in a cooperative task with a human.

Not ideal if you are looking for a pre-built game for general entertainment or a tool to simply deploy a conversational agent without evaluating its human-AI teaming performance.

AI-evaluation human-AI-interaction conversational-AI visual-question-answering crowdsourcing-experiments
No License Stale 6m No Package No Dependents
Maintenance 0 / 25
Adoption 6 / 25
Maturity 8 / 25
Community 16 / 25

How are scores calculated?

Stars

23

Forks

6

Language

Lua

License

Last pushed

Nov 22, 2022

Commits (30d)

0

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/agents/GT-Vision-Lab/GuessWhich"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.