eric-ai-lab/MSSBench
[ICLR 2025] Official codebase for the ICLR 2025 paper "Multimodal Situational Safety"
This project helps evaluate multimodal AI models to determine if they can identify and avoid unsafe situations based on images and instructions. It takes in multimodal AI models, a dataset of images and associated safe/unsafe descriptions or instructions, and outputs a safety score or evaluation for the models. It is designed for AI safety researchers and developers working on responsible AI.
No commits in the last 6 months.
Use this if you are developing or evaluating AI systems that interact with users through images and text and need to ensure they operate safely.
Not ideal if you are looking for a pre-built safety filter or an AI model to directly apply to a production system.
Stars
30
Forks
2
Language
Python
License
MIT
Category
Last pushed
Jun 23, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/agents/eric-ai-lab/MSSBench"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
google-deepmind/concordia
A library for generative social simulation
Mai-xiyu/Minecraft_AI
AI Play Minecraft
mikelma/craftium
A framework for creating rich, 3D, Minecraft-like single and multi-agent environments for AI...
cocacola-lab/MineLand
Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs
rezaho/MARSYS
Multi-Agent Reasoning Systems