demml/scouter
Monitoring, Evaluation and Observability for AI Applications
This tool helps machine learning engineers and AI developers ensure their deployed AI applications and models are performing as expected. It monitors the quality of incoming data, tracks model performance metrics, and evaluates AI agent behavior in production. You feed it your model's input and output data, and it provides real-time alerts and detailed insights into potential issues.
Use this if you need a comprehensive, single solution for monitoring the health and performance of your production machine learning models and AI agents.
Not ideal if you are looking for a general-purpose application monitoring tool not specifically focused on AI/ML workflows.
Stars
11
Forks
—
Language
Rust
License
MIT
Category
Last pushed
Mar 13, 2026
Monthly downloads
44
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/demml/scouter"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Arize-ai/openinference
OpenTelemetry Instrumentation for AI Observability
vndee/llm-sandbox
Lightweight and portable LLM sandbox runtime (code interpreter) Python library.
apache/hertzbeat
An AI-powered next-generation open source real-time observability system.
traceloop/openllmetry
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
utkuozdemir/nvidia_gpu_exporter
Nvidia GPU exporter for prometheus using nvidia-smi binary