opendatalab/UrBench
[AAAI 2025]This repo contains evaluation code for the paper “UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios”
This tool helps urban planners, smart city developers, and researchers assess how well AI models understand complex urban environments. It takes in various urban image data, like street views and satellite images, and evaluates AI models on tasks ranging from urban planning analysis to everyday city issues, providing scores for their performance. This is for professionals and researchers working on AI applications for urban development.
No commits in the last 6 months.
Use this if you need a comprehensive and challenging way to benchmark large multimodal AI models specifically for multi-view urban scenarios.
Not ideal if you are evaluating general-purpose AI models or models for non-urban-specific tasks.
Stars
36
Forks
5
Language
Python
License
—
Category
Last pushed
Apr 10, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/opendatalab/UrBench"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
TsinghuaC3I/MARTI
A Framework for LLM-based Multi-Agent Reinforced Training and Inference
zjunlp/KnowLM
An Open-sourced Knowledgable Large Language Model Framework.
cli99/llm-analysis
Latency and Memory Analysis of Transformer Models for Training and Inference
tanyuqian/redco
NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to...
stanleylsx/llms_tool
一个基于HuggingFace开发的大语言模型训练、测试工具。支持各模型的webui、终端预测，低参数量及全参数模型训练(预训练、SFT、RM、PPO、DPO)和融合、量化。