MAC-AutoML/SocialOmni
Benchmarking Audio-Visual Social Interactivity in Omni Models
This project helps evaluate how well AI models can participate in natural, multi-person conversations, especially in video calls or real-world interactions. It takes audio-visual data of people talking and measures if a model understands who is speaking, when to interject naturally, and how to respond appropriately. This is for researchers and developers who are building or improving AI models that engage in complex social dialogue.
Use this if you are developing or benchmarking large language models that need to understand and participate in dynamic, multi-speaker audio-visual conversations with natural timing and content.
Not ideal if you are looking for a tool to analyze existing human conversations or for simple, static question-and-answer evaluations of AI models.
Stars
17
Forks
—
Language
Python
License
—
Category
Last pushed
Mar 18, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/MAC-AutoML/SocialOmni"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
MMMU-Benchmark/MMMU
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal...
pat-jj/DeepRetrieval
[COLM’25] DeepRetrieval — 🔥 Training Search Agent by RLVR with Retrieval Outcome
lupantech/MathVista
MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts
x66ccff/liveideabench
[𝐍𝐚𝐭𝐮𝐫𝐞 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬] 🤖💡 LiveIdeaBench: Evaluating LLMs' Scientific Creativity and Idea...
ise-uiuc/magicoder
[ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct