Guides¶
Each guide takes one task from start to finish. They are for people who have run the quickstart and want to do something specific.
| Guide | What it covers |
|---|---|
| Providers | every model target, the key it reads, and logged-in Claude Code and Codex as judge |
| Coding agents | the AGENTS.md section and the skill init writes, so a coding agent can use evalproof in your repository |
| Testing agents | traces, the checks scorer, several turns and variants |
| Testing a system in any language | run a Node.js, Go or shell program as the system, with the case on stdin |
| Testing a TypeScript or JavaScript system | the npm helper: runCommand, serve, a trace from a Vercel AI SDK or LangChain.js result, init --node, and scan for TypeScript tools and agents |
| Repeated trials | run each case several times, and compare several candidates in one table |
| Readiness | which parts of quality a suite covers and which it leaves untested |
| Find what is not tested | scan a repository for model calls, agents and tools no suite tests, and draft inputs for them, never answers |
| Agent SDK adapters | build a trace from ADK, LangChain/LangGraph, the Claude Agent SDK, OpenAI Agents, LlamaIndex, CrewAI, Pydantic AI, DSPy, Haystack, AutoGen or Semantic Kernel |
| Import results from other tools | gate promptfoo, Inspect, DeepEval, lm-eval, garak, ADK, Claude Code plugin eval, CSV and JSONL results, or turn an ADK eval set into a dataset |
| Import test cases from other tools | turn the test cases you already have in promptfoo, DeepEval, Ragas, LangSmith, OpenAI Evals, Inspect, Braintrust, Langfuse, Phoenix, JSONL or CSV into a dataset and a suite, or start a repository with init --from |
| From Python | the same functions the CLI uses |
| In CI | the GitHub Action and the workflows init writes |
| Troubleshooting | what an error message means and how to fix it |