gordicaleksa/serbian-llm-eval
Serbian LLM Eval.
This project helps evaluate how well large language models (LLMs) understand and generate Serbian text. It takes an LLM (such as one you've developed or are considering using) as input and provides performance scores across various tasks like common sense reasoning, world knowledge, and reading comprehension. This is useful for researchers and developers working on building or deploying LLMs tailored for the Serbian language and potentially other Serbo-Croatian-Bosnian (HBS) languages.
No commits in the last 6 months.
Use this if you need to objectively measure the capabilities of an LLM in Serbian, specifically for tasks like understanding context, answering questions based on general knowledge, and comprehending text.
Not ideal if you are looking to evaluate an LLM's performance in complex areas like advanced mathematics, coding, or aggregated benchmarks like MMLU and BBH, as these are not yet covered.
Stars
97
Forks
8
Language
Python
License
—
Category
Last pushed
Mar 19, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/gordicaleksa/serbian-llm-eval"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
EvolvingLMMs-Lab/lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
vibrantlabsai/ragas
Supercharge Your LLM Application Evaluations 🚀
open-compass/VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
EuroEval/EuroEval
The robust European language model benchmark.
Giskard-AI/giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents