THU-KEG/COPEN
The official code and dataset for EMNLP 2022 paper "COPEN: Probing Conceptual Knowledge in Pre-trained Language Models".
This project provides a benchmark to evaluate how well Pre-trained Language Models (PLMs) understand concepts, not just words. It takes a PLM and a set of conceptual tasks (like judging similarity or properties) as input. The output helps researchers understand if their PLM can grasp human-like conceptual knowledge. This is for AI researchers or language model developers who want to analyze and improve their models' cognitive abilities.
No commits in the last 6 months.
Use this if you are an AI researcher or developer working on Pre-trained Language Models and need a standardized way to test their conceptual understanding, beyond basic linguistic tasks.
Not ideal if you are looking for a tool to directly apply language models for downstream applications like text generation, summarization, or translation.
Stars
21
Forks
1
Language
Python
License
MIT
Category
Last pushed
Mar 09, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/THU-KEG/COPEN"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ymcui/cmrc2018
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (CMRC 2018)
thunlp/MultiRD
Code and data of the AAAI-20 paper "Multi-channel Reverse Dictionary Model"
princeton-nlp/DensePhrases
[ACL 2021] Learning Dense Representations of Phrases at Scale; EMNLP'2021: Phrase Retrieval...
IndexFziQ/KMRC-Papers
A list of recent papers regarding knowledge-based machine reading comprehension.
danqi/rc-cnn-dailymail
CNN/Daily Mail Reading Comprehension Task