Skip to content

Guides

Each guide takes one task from start to finish. They are for people who have run the quickstart and want to do something specific.

Guide What it covers
Providers every model target, the key it reads, and logged-in Claude Code and Codex as judge
Coding agents the AGENTS.md section and the skill init writes, so a coding agent can use evalproof in your repository
Testing agents traces, the checks scorer, several turns and variants
Testing a system in any language run a Node.js, Go or shell program as the system, with the case on stdin
Testing a TypeScript or JavaScript system the npm helper: runCommand, serve, a trace from a Vercel AI SDK or LangChain.js result, init --node, and scan for TypeScript tools and agents
Repeated trials run each case several times, and compare several candidates in one table
Readiness which parts of quality a suite covers and which it leaves untested
Find what is not tested scan a repository for model calls, agents and tools no suite tests, and draft inputs for them, never answers
Agent SDK adapters build a trace from ADK, LangChain/LangGraph, the Claude Agent SDK, OpenAI Agents, LlamaIndex, CrewAI, Pydantic AI, DSPy, Haystack, AutoGen or Semantic Kernel
Import results from other tools gate promptfoo, Inspect, DeepEval, lm-eval, garak, ADK, Claude Code plugin eval, CSV and JSONL results, or turn an ADK eval set into a dataset
Import test cases from other tools turn the test cases you already have in promptfoo, DeepEval, Ragas, LangSmith, OpenAI Evals, Inspect, Braintrust, Langfuse, Phoenix, JSONL or CSV into a dataset and a suite, or start a repository with init --from
From Python the same functions the CLI uses
In CI the GitHub Action and the workflows init writes
Troubleshooting what an error message means and how to fix it