프로젝트 스크린샷
개요
promptfoo is a developer-first tool designed to replace trial-and-error prompt engineering with data-driven evaluation. It allows developers to define test cases in simple declarative configuration files, run evaluations against multiple LLM providers simultaneously, and visualize results through a local web viewer or command-line interface. Beyond functional evaluation, it includes red teaming and vulnerability scanning capabilities to help secure AI applications against adversarial inputs. The tool runs evaluations 100% locally, ensuring prompts and data do not leave the developer's machine. It is actively used in production environments and is now part of OpenAI while remaining MIT licensed and open source.
주요 기능
- CLI and library for LLM evaluation and red teaming
- Declarative configuration for test cases and prompts
- Side-by-side model comparison
- Red teaming and vulnerability scanning with report generation
- Local web viewer and command-line output
- CI/CD integration and code scanning for pull requests
- 100% local evaluation execution for privacy
- Live reload and caching for fast iteration
- Supports multiple LLM providers and APIs
요구 사항, 설치 및 빠른 시작
사용 정보
모델 호환성 및 사용 사례
Supports comparison across multiple LLM providers including OpenAI, Anthropic, Azure, Bedrock, Ollama, and others. A full list of supported models is available in the provider documentation.
라이선스 및 위험 참고 사항
Licensed under the MIT License. The project is now part of OpenAI but remains open source under the same license.
Editorial verification 2026-08-09: repository URL, owner, description, license and repository statistics were reviewed. License metadata: MIT. README was fetched for the channel draft; re-check repository dependencies, releases and model terms before production use.
릴리스 및 유지 관리
Not stated in the repository metadata.