Promptfoo

Promptfoo provides an open-source CLI and library for evaluating and red-teaming LLM applications. Its documentation describes configuration-driven test cases, assertions, provider comparisons and CI integration. The repository's license permits use and modification under MIT terms; that license applies to the published software, not every feature of the commercial platform. The company's about page states that Promptfoo is now part of OpenAI.

For a developer, the practical decision is who will maintain the test cases and judge failures. Promptfoo can make checks repeatable, but the team still has to choose representative inputs, useful assertions and acceptable failure thresholds. A passing run is evidence about those configured cases, not proof that an application is safe against every attack or accurate on all future requests. Running the CLI locally also does not make a remote model request local: review each selected provider and the data it receives.

The pricing page separates Community from Enterprise and On-Premise. Community includes local evaluation and some red-team usage; commercial plans add controls such as team access, managed deployment and support. Compare the required controls and probe limits before choosing a tier. Budget model/API calls, runner infrastructure and test maintenance separately. This directory read the documents without configuring an evaluation or scanning a target, so it makes no claim about detection rate, false positives, runtime or comparative dashboard quality.