Nativeatom/FRoG
Fuzzy reasoning of Generalized Quantifiers (EMNLP 2024)
FRoG helps evaluate how well language models understand and reason with 'fuzzy' percentage terms like 'a small amount' or 'moderate' in real-world math problems. You input a math problem where a percentage is hidden, and a language model tries to select the most fitting fuzzy quantifier from a set of choices. This is used by researchers and developers who are building or testing large language models.
No commits in the last 6 months.
Use this if you are a researcher or developer who needs to assess the nuanced reasoning abilities of large language models, particularly their comprehension of imprecise numerical concepts.
Not ideal if you're looking for a tool to solve math problems directly or to convert fuzzy language into precise percentages for non-AI applications.
Stars
8
Forks
—
Language
Python
License
Apache-2.0
Category
Last pushed
Jan 12, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/Nativeatom/FRoG"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ymcui/cmrc2018
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (CMRC 2018)
thunlp/MultiRD
Code and data of the AAAI-20 paper "Multi-channel Reverse Dictionary Model"
princeton-nlp/DensePhrases
[ACL 2021] Learning Dense Representations of Phrases at Scale; EMNLP'2021: Phrase Retrieval...
IndexFziQ/KMRC-Papers
A list of recent papers regarding knowledge-based machine reading comprehension.
danqi/rc-cnn-dailymail
CNN/Daily Mail Reading Comprehension Task