fajri91/discourse_probing
Discourse Probing of Pretrained Language Models. In Proceedings of NAACL 2021.
This project helps natural language processing researchers evaluate how well different language models understand the overall structure and meaning of documents, beyond just individual words or sentences. It takes a pretrained language model and text data, then outputs an assessment of the model's ability to handle tasks like predicting the next sentence, ordering shuffled sentences, or identifying discourse markers. NLP researchers and scientists would use this to compare and benchmark language models for their understanding of document-level relationships.
No commits in the last 6 months.
Use this if you are an NLP researcher interested in systematically evaluating the discourse understanding capabilities of various pretrained language models across different tasks and languages.
Not ideal if you are looking for a tool to directly apply language models to solve a specific business problem, rather than to analyze their internal mechanisms.
Stars
10
Forks
1
Language
Jupyter Notebook
License
—
Category
Last pushed
Jun 27, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/fajri91/discourse_probing"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ymcui/cmrc2018
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (CMRC 2018)
princeton-nlp/DensePhrases
[ACL 2021] Learning Dense Representations of Phrases at Scale; EMNLP'2021: Phrase Retrieval...
thunlp/MultiRD
Code and data of the AAAI-20 paper "Multi-channel Reverse Dictionary Model"
IndexFziQ/KMRC-Papers
A list of recent papers regarding knowledge-based machine reading comprehension.
danqi/rc-cnn-dailymail
CNN/Daily Mail Reading Comprehension Task