DFKI-NLP/gevalm
Code and data for the paper "Evaluating German Transformer Language Models with Syntactic Agreement Tests" (Zaczynska et al., 2020)
This project helps evaluate how well German AI language models understand grammar, specifically syntactic agreement. It takes in test sentences designed to check grammatical rules and outputs accuracy scores, indicating how often the models correctly apply those rules. Developers and researchers working with German natural language processing models would use this to assess and compare different models' linguistic capabilities.
No commits in the last 6 months.
Use this if you need to quantitatively measure the grammatical understanding of German transformer language models, especially regarding syntactic agreement.
Not ideal if you are looking to train a new language model or evaluate models in languages other than German.
Stars
7
Forks
2
Language
Python
License
—
Category
Last pushed
Jun 12, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/DFKI-NLP/gevalm"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
LoicGrobol/zeldarose
Train transformer-based models.
CPJKU/wechsel
Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of...
yuanzhoulvpi2017/zero_nlp
中文nlp解决方案(大模型、数据、模型、训练、推理)
minggnim/nlp-models
A repository for training transformer based models
IntelLabs/nlp-architect
A model library for exploring state-of-the-art deep learning topologies and techniques for...