claws-lab/multimodal-robustness
Code and resources for EMNLP 2022 paper on 'Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutions'
This project helps evaluate how robust multimodal AI classifiers are when presented with 'diluted' information, where crucial details might be missing from either images or text. It takes in images and text, extracts key information from both, and then generates modified, 'diluted' versions to test the AI's ability to maintain correct classifications. This tool is for AI researchers and practitioners working on systems that combine information from different sources, like images and text, to make decisions or classify data.
No commits in the last 6 months.
Use this if you need to understand how well your multimodal classification AI performs when one of its input channels (like images or text) contains less relevant information or is 'diluted'.
Not ideal if you are looking for a general-purpose, off-the-shelf multimodal classifier for direct deployment in a production environment without needing to test its robustness to diluted content.
Stars
10
Forks
1
Language
Python
License
—
Category
Last pushed
Mar 11, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/claws-lab/multimodal-robustness"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
thunlp/OpenAttack
An Open-Source Package for Textual Adversarial Attack.
thunlp/TAADpapers
Must-read Papers on Textual Adversarial Attack and Defense
jind11/TextFooler
A Model for Natural Language Attack on Text Classification and Inference
thunlp/OpenBackdoor
An open-source toolkit for textual backdoor attack and defense (NeurIPS 2022 D&B, Spotlight)
thunlp/SememePSO-Attack
Code and data of the ACL 2020 paper "Word-level Textual Adversarial Attacking as Combinatorial...