disi-unibo-nlp/nlg-metricverse
[COLING22] An End-to-End Library for Evaluating Natural Language Generation
When developing or researching Natural Language Generation (NLG) models, it's crucial to accurately assess their output. This tool takes your model's generated text and a set of human-written reference texts, then calculates various automatic evaluation scores. It's designed for NLP researchers and engineers who need to understand and compare the quality of different NLG systems, such as those used for summarization, translation, or chatbots.
No commits in the last 6 months. Available on PyPI.
Use this if you are developing or fine-tuning NLG models and need a comprehensive, consistent way to evaluate their performance using a wide array of automatic metrics.
Not ideal if you primarily rely on human evaluation or only need a single, basic metric for a well-established NLG task.
Stars
94
Forks
5
Language
Python
License
MIT
Category
Last pushed
Dec 18, 2023
Commits (30d)
0
Dependencies
26
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/disi-unibo-nlp/nlg-metricverse"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
google/langfun
OO for LLMs
tanaos/artifex
Small Language Model Inference, Fine-Tuning and Observability. No GPU, no labeled data needed.
preligens-lab/textnoisr
Adding random noise to a text dataset, and controlling very accurately the quality of the result
vulnerability-lookup/VulnTrain
A tool to generate datasets and models based on vulnerabilities descriptions from @Vulnerability-Lookup.
masakhane-io/masakhane-mt
Machine Translation for Africa