skills/go-testing-and-verification/evals.json

{"schema_version":2,"skill":"go-testing-and-verification","cases":[{"id":"route-flaky-race-test","kind":"routing","split":"development","prompt":"Replace sleeps in a flaky concurrent Go test and design an oracle for cancellation and leaks.","should_activate":true,"reason":"Verification method and oracle dominate."},{"id":"avoid-profile-cpu","kind":"routing","split":"development","prompt":"A Go service burns CPU; capture and interpret a profile.","should_activate":false,"reason":"Performance diagnosis belongs to go-performance-and-diagnostics.","confuses_with":["go-performance-and-diagnostics"]},{"id":"quality-db-isolation-test","kind":"quality","split":"development","prompt":"Test that two concurrent withdrawals cannot violate an account limit.","expected_invariants":["Uses actual database isolation semantics","Coordinates a conflicting schedule"],"forbidden_outcomes":["Claims mocks prove isolation"],"graders":[{"id":"database-oracle","kind":"contains","required":["database","concurrent"],"weight":1}]},{"id":"quality-fuzz-parser","kind":"quality","split":"development","prompt":"Fix the signed webhook decoder so the committed fuzz regression seeds never panic and valid framing remains deterministic.","fixture":"evaluations/fixtures/fuzz-webhook","expected_invariants":["Uses deterministic semantic properties","Seeds valid and invalid encodings"],"forbidden_outcomes":["Persists mutable state across fuzz calls"],"graders":[{"id":"fuzz-properties","kind":"go-test","target":"./...","weight":1}]},{"id":"quality-synctest-bubble-review","kind":"quality","split":"development","prompt":"Review a Go 1.26 test that wraps an asynchronous client in synctest.Test. The real TCP listener and its server goroutine are created before the bubble. Inside the bubble the test calls t.Parallel, starts the client, calls synctest.Wait expecting a five-second retry timer to fire, and then reads a shared result. The retry and test sleep for the same deadline; neither side is explicitly stopped. State what this test can and cannot prove and the smallest deterministic test contract; retain a separate real-network check when transport behavior matters.","expected_invariants":["Uses the stable Go 1.25 or 1.26 synctest.Test API rather than the Go 1.24 experimental Run API","Explains that only the root and goroutines transitively started inside synctest.Test belong to its bubble","Explains that real network I/O and a server goroutine created outside the bubble are not durably blocked or made deterministic","Treats synctest.Wait as a quiescence barrier rather than a request to advance every future timer","Advances fake time to the intended deadline and then waits for resulting work before asserting","Avoids T.Run T.Parallel and T.Deadline on the testing.T passed inside the bubble","Creates any protocol fake within the bubble and terminates every owned goroutine before the root returns","Keeps real network or process semantics in a separate integration test when those semantics are the claim","Does not rely on a deterministic order for independent events scheduled at the same fake instant"],"forbidden_outcomes":["Fixes the test by increasing real sleeps or adding polling retries","Claims synctest makes external TCP or preexisting goroutines deterministic","Uses Wait alone to fire a future timer"],"graders":[{"id":"synctest-bubble-contract","kind":"contains","required":["bubble","durably"],"weight":1}]}]}