evals/redesign-routing-heldout-v2/graders/validation.md

---
type: llm
---
PASS if the answer includes all three required elements below, in any wording or method. Parenthetical examples are illustrative, not required.
(1) A concrete check of the new flow involving lapsed members (for example, an uncoached task, moderated sessions, or an unmoderated test with recently lapsed members).
(2) An observable result from that check that would show the new flow works (for example, members can say when their benefits start without help, or complete renewal without wrong turns). Post-launch aggregates such as support questions on the two topics or exits compared with the supplied 41% may supplement this result but cannot replace it.
(3) No invented targets, timings, or findings. Using the supplied 41% as a baseline, or choosing how many sessions to run, is not an invented finding.
FAIL if validation is missing, names no observable result from a check with lapsed members, relies only on support volume or other post-launch aggregates, or cites findings, benchmarks, or thresholds that were not supplied.