sarva-20/LLM-Observability-FOSS
🧠Learn LLM Observability step-by-step using FOSS tools. From zero visibility to full monitoring with Langtrace, OpenTelemetry, and Jaeger. Demo from FOSS United Coimbatore Meetup.
Building an application powered by large language models (LLMs) can feel like a black box, making it hard to understand costs, debug issues like slow responses or incorrect answers, and monitor performance. This project helps you add 'observability' to your LLM application, turning a black box into a transparent system. It takes your LLM application's code and, through open-source tools, produces detailed insights into its operations, like token usage, failure points, and response times. Developers and engineers building and managing LLM-powered applications will use this to gain crucial operational visibility.
Use this if you are a developer or engineer building an LLM application and need to understand its internal workings, costs, and performance using free and open-source tools.
Not ideal if you are looking for a fully managed, commercial observability solution or if your application does not involve large language models.
Stars
41
Forks
1
Language
Python
License
MIT
Category
Last pushed
Jan 07, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/sarva-20/LLM-Observability-FOSS"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Arize-ai/openinference
OpenTelemetry Instrumentation for AI Observability
apache/hertzbeat
An AI-powered next-generation open source real-time observability system.
traceloop/openllmetry
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
utkuozdemir/nvidia_gpu_exporter
Nvidia GPU exporter for prometheus using nvidia-smi binary
Dynatrace/obslab-llm-observability
Search for a holiday and get destination advice from an LLM. Observability by Dynatrace.