Where resources.html is a broad reference library, this page is our curated list of specific tools for monitoring, workflow automation, and evaluation. It's not comprehensive — it's signal.
Tools for observability, tracing, and catching model degradation in production.
Best open-source option for LLM tracing when you need self-hosted control. Traces, prompt versions, scores — all in one dashboard.
Go-to for teams already on OpenTelemetry. Embedding drift detection catches model degradation before users do.
Vendor-agnostic OTel instrumentation. If you're building on multiple providers, this is the tracing layer that doesn't lock you in.
Best for teams that need LLM + traditional ML monitoring in one place. The text descriptor system is underused by most teams.
Tools for intelligent routing, cost control, and managing inference endpoints.
Single API endpoint for 100+ providers. The budget limit enforcement alone has saved teams from surprise $40K bills.
The standard for self-hosted inference. PagedAttention is the reason GPU cost drops by 3-4x on real workloads.
Best real-time model quality-to-price comparison. Run this before every model selection decision.
One-line integration for OpenAI cost tracking by team and feature. Worth it for any team spending >$2K/month on inference.
Enter your email to instantly unlock our top picks for Agent Frameworks, Evaluation, RAG, and Workflow Automation.