Agent Evals

A platform-agnostic agent evaluation library. Call run_eval / run_eval_async with a task, dataset, scorers, and a platform string ("local", "braintrust", "mlflow", "langfuse"); adapters handle everything platform-specific. For breadth-coverage benchmarking across many single-turn scenarios sliced by capability, subclass agent_evals.BaseAgent and call run_benchmark_async.

This is a library, not an application — you depend on its public API from your own code.

• Quickstart — install and run your first eval
• Architecture — hexagonal design, ports and adapters, rationale
• Benchmarks — YAML benchmark schema and run_benchmark_async
• Scorers — native, Autoevals, and AgentEvals scorers
• Platforms — Braintrust, MLflow, LangFuse, and local adapters
• Converters — framework message normalizers
• Writing a new adapter — step-by-step guide