FAILURECASE ↗

AI-agent eval writer · Free sample

Turn “what if?” into an agent eval.

Start with five refund and cancellation patterns. Shape the request, fixture and checks around your policy, then export cases for your own agent test setup.

No model calls or uploads. Download a project to keep your work, or choose local recovery on this device. Cases are synthetic drafts for policy review.

Keep your work

Save the whole workspace, including unfinished edits. JSONL exports below remain ready for your agent test setup.

Optional local recovery saves policy fields and cases in this browser profile. Anyone using this profile may access them. Observations and reports are excluded; no data is uploaded. Keep a project file as a separate backup.

Working in memory. Local recovery is off.

1. Set your support policy

10000 = 100.00 in the selected currency. The last eligible day and automatic amount limit are inclusive.

2. Write and review your evals

Choose a case. Edit its scenario and checks, save it, then review the policy assumptions.

Starting fixture

Synthetic sandbox data for this scenario. Values keep their chosen type, including numbers, booleans and null.

Checks for observed behavior

These check the state and tool events captured by your adapter. Edit values and types; review the resulting checks before using them.

Advanced: edit case JSON

Save or discard guided edits first. JSON and the guided editor share the same saved case.

Approval records your policy review. Editing the scenario or checks resets it. Agent behavior still needs separate observation and review.

3. Check your agent’s observed behavior

Run the cases using your own agent and sandbox adapter, then load its observations here. The writer does not run your agent. Review response quality and other activity separately.

No observations loaded.

Full grading report JSON
Your report will appear here.

Keep going with all 32 patterns, the offline writer, Node grader and incident-to-eval workflow. See Team and Agency editions →