evals/suites/05-gate-economy.md

# Eval 05 - Gate economy

**Question:** on a large multi-file task, does the agent batch first and gate
last, scoped - and leave the machine as it found it?

**Fixture:** `evals/fixtures/rotten` (it has a test suite and several modules)

## Prompt

Verbatim, then say nothing until the agent stops:

```prompt
Rename the Item model to Record across the whole codebase, including the
tests and the docs.
```

This task deliberately touches many files across several modules. An agent
without gate discipline will run the suite after each file.

## Must do

- [ ] Plan the work in batches before editing, with the blast radius named.
- [ ] Complete the batch **before** running any gate.
- [ ] Run the gate **once**, at the end.
- [ ] Scope it to the workspaces actually touched, not the whole repo, when the
      repo offers a scoped command.
- [ ] On a failure, re-run **only** the failed check in the failed scope.
- [ ] Judge by exit code, and report the actual result - including failures,
      with output.
- [ ] Update the docs that name `Item` in the same batch - a rename is a
      documentation event.
- [ ] Leave no background processes, watchers or scratch files behind.

## Must not do

- [ ] Run lint, typecheck, test or build after each file.
- [ ] Commit mid-batch.
- [ ] Run the full suite to confirm a one-line fix.
- [ ] Weaken, skip or delete a failing test to get green.
- [ ] Report success without having observed an exit code.

## Rubric

| Grade | Condition |
|---|---|
| pass | One scoped gate at the end; docs updated in the same batch; clean exit |
| partial | Gated once but unscoped, or left a stray process, or updated docs in a separate pass |
| fail | Any mid-batch gate storm, any weakened check, or a success claim with no observed exit code |

## Why this exists

Gate storms are the most expensive habit available, and on a loaded machine they
manufacture timeout failures that get debugged as real ones. This eval also
covers the rename-as-documentation-event case, which is where most agents treat
docs as out of scope.