sinanuozdemir/oreilly-evaluating-llms
Metrics, Benchmarks, and Practical Tools for Assessing Large Language Models
This project provides practical tools and techniques for understanding how well large language models (LLMs) perform. You can assess metrics like text quality, classification accuracy, and factual recall to see if a model meets your specific needs. It's for anyone building, deploying, or managing AI systems who needs to ensure their LLMs are reliable and effective.
No commits in the last 6 months.
Use this if you are a machine learning engineer, data scientist, or product manager who needs to rigorously evaluate different large language models for various tasks, from generating text to understanding user intent.
Not ideal if you are looking for a simple plug-and-play solution without needing to understand the underlying evaluation methodologies or customize assessment metrics.
Stars
26
Forks
17
Language
—
License
—
Category
Last pushed
Feb 16, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/sinanuozdemir/oreilly-evaluating-llms"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
EvolvingLMMs-Lab/lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
vibrantlabsai/ragas
Supercharge Your LLM Application Evaluations 🚀
open-compass/VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
EuroEval/EuroEval
The robust European language model benchmark.
Giskard-AI/giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents