Promptfoo

What is Promptfoo?

Promptfoo is a powerful, open-source Command Line Interface (CLI) and library, designed to evaluate and enhance the output quality of Language Learning Models (LLMs). It enables users to systematically test prompts, compare results side-by-side, and score outputs automatically, all while prioritizing privacy by running locally on your machine. With its language-agnostic design, it's perfect for use with Python, Javascript, or any other language. Whether you're comparing the outputs of GPT 3 and GPT 4, or evaluating the impact of adding adjectives to a bot's personality, Promptfoo makes it simple and efficient. Plus, its built-in share functionality and web viewer make collaboration a breeze.