AI Monitoring & Observability

Open-source LLM observability and evaluation. Trace visualization, embedding analysis, and evals.

Enterprise AI evaluation and observability platform. Prompt playground, scoring, and dataset management.

LLM monitoring within Datadog ecosystem. Trace prompts, tokens, latency alongside infrastructure metrics.

Open-source LLM evaluation framework. 14+ metrics including hallucination, relevancy, and bias detection.

LLM evaluation and hallucination detection platform. Automated metrics for RAG quality and safety.

Open-source LLM observability proxy. One-line integration, request logging, caching, and rate limiting.

Prompt management and evaluation platform. Version prompts, run evals, and optimize LLM performance.

Open-source LLM observability. Traces, metrics, prompt management, and evaluation. Self-hostable.

LangChain's observability platform. Trace, debug, and evaluate LLM applications with detailed run analytics.

Open-source observability for LLMs based on OpenTelemetry. Works with Datadog, Grafana, Honeycomb.

Open-source LLM evaluation and tracing platform. Track experiments, evaluate outputs, and debug prompts.

AI gateway with observability. Load balancing, fallbacks, caching, and guardrails for LLM APIs.

Open-source product analytics with AI feature tracking. Session replay, feature flags, A/B testing.

Prompt engineering platform. Version control, A/B testing, and analytics for prompts across providers.

Evaluation framework for RAG pipelines. Measures faithfulness, relevancy, and context precision.

Error tracking with AI/LLM monitoring support. Track exceptions, performance, and LLM-specific errors.

LLM monitoring built on OpenTelemetry. Auto-instrumentation for LangChain, LlamaIndex, and OpenAI SDK.

ML experiment tracking and model monitoring. LLM-specific features for prompt tracking and evaluation.