Arize AI

Arize sells observability and evaluation for LLM applications and agents, with tracing, evals and experimentation under one roof and an open-source instrumentation layer feeding it. It also offers an open-source observability and evals product alongside the managed platform, which is worth knowing because it means the instrumentation you adopt is not locked to the paid tier.

The problem it addresses is real and specific: a conventional request log tells you an agent failed, not which of the ten steps inside it failed, why the retrieved context was wrong, or whether the change you just shipped made things better or worse. Tracing gives you the step, evaluation gives you the judgement, and you need both — tracing alone tells you what happened without saying whether it was good, and evals alone tell you the score without showing the path that produced it. As with every platform in this category, the vendor's own benchmark and improvement claims are the vendor's.