Know whether the next version is better.

Agent Evaluation. Compare agent releases, models, and instructions on the same cases. Inspect regressions, bring in human judgment, and carry the evidence into your release decision.

Explore agent evaluation

Native product components with illustrative support data.

Pascal Evaluation comparing support cases across two Agent Releases with illustrative data.. Native Pascal UI product preview. — Product preview

Agent Evaluation. A clear case for what to change.

Build useful datasets

Author or import cases and curate captured failures. Keep reference answers, source lineage, and dataset versions available for review.

Define what good means

Combine deterministic checks, Python scorers, model judges, and classifiers. Publish exact evaluator versions and test them on examples.

Compare variants in a playground

Run the same cases across models, instructions, or exact Agent Releases. Inspect outputs and scores, then rerun the cases that need attention.

Look beyond an average

Compare paired outputs and relevant slices. Review uncertainty, regressions, and missing results before accepting a candidate.

Bring in human judgment

Use independent reviews and adjudication to resolve disagreements. Compare judge results with accepted human labels and inspect calibration coverage.

Turn failures into regression cases

Explore behavior distributions and failure groups. Review examples, retain accepted expectations, and publish cases for the next evaluation.

Explore Pascal

One failure. A better next version.

Follow a support agent’s unverified refund promise from captured evidence to a reviewed case and a comparison of two releases.

Locate the captured refund promise and the calls around it.

Find the failure. Native product view with illustrative data.

In practice

The support team compares releases 3 and 4 against refund cases. The candidate requests approval on the disputed case, but another planned result is missing. The team keeps the comparison inconclusive and collects more evidence before making a release decision.

Example operation.Explore the solution

Agent Evaluation in your operation.

Connect it to the rest of your work.

Start with captured evidence in Observability. Bring in independent human review through Lab. Carry the evaluation into the next agent draft and release decision.

What you work with

Agent Evaluation

Versioned datasets and frozen evaluation samples
Evaluator versions and exact agent variants
Runs, paired comparisons, and human review

Stay in control as the work moves.

A comparison retains its exact cases, evaluators, and targets. Missing results remain unscored, and a favorable example does not authorize a release.

Where you work

Agent Evaluation

Dataset and evaluator workbenches
Experiment playground and case inspection
Comparison decisions and regression preparation

Make evaluation part of development.

Work with the same cases and versions.

Use TypeScript and Python interfaces to work with datasets, evaluators, runs, and comparisons.

Request developer docs

Bring the decision into CI.

The CLI can check an exact retained evaluation decision and matching inputs. Publishing and activating an Agent Release remain separate steps.

Request CLI docs

Give the next release a better test.

Choose an agent, the failures that matter, and the criteria your team would trust to judge an improvement.

Explore agent evaluation Explore Agent Observability