skills/go-performance-and-diagnostics/evals.json

{"schema_version":2,"skill":"go-performance-and-diagnostics","cases":[{"id":"route-tail-latency","kind":"routing","split":"development","prompt":"Diagnose rising P99 latency, allocations, and mutex wait in this Go service before optimizing.","should_activate":true,"reason":"Measured resource diagnosis dominates."},{"id":"avoid-overload-policy","kind":"routing","split":"development","prompt":"Design retry budgets and load shedding for a failing dependency.","should_activate":false,"reason":"Feedback-system resilience belongs to go-service-resilience.","confuses_with":["go-service-resilience"]},{"id":"quality-profile-choice","kind":"quality","split":"development","prompt":"RSS rises while heap in-use looks flat and goroutines grow.","expected_invariants":["Checks goroutine stacks and non-heap memory","Avoids assuming a heap leak"],"forbidden_outcomes":["Adds pooling before finding retention"],"graders":[{"id":"diagnostic-evidence","kind":"contains","required":["goroutine","profile"],"weight":1}]},{"id":"quality-benchmark","kind":"quality","split":"development","prompt":"Prove a parser optimization improves throughput without hidden allocation or semantic cost.","expected_invariants":["Compares identical semantics over repeated runs","Reports effect and allocation cost"],"forbidden_outcomes":["Uses one benchmark sample"],"graders":[{"id":"benchmark-evidence","kind":"contains","required":["allocations","compare"],"weight":1}]},{"id":"quality-memory-budget-review","kind":"quality","split":"development","prompt":"Review a Go 1.26 service after a container-memory incident. The pod hard limit is 1 GiB. Process RSS reaches 910 MiB while the in-use heap profile is 260 MiB and `/memory/classes/total:bytes - /memory/classes/heap/released:bytes` is 540 MiB. The process uses a C image library and `syscall.Mmap`. A patch sets `GOMEMLIMIT=1GiB`, calls `debug.FreeOSMemory()` every second, and declares the OOM fixed because the live heap is flat. State the diagnostic and memory-budget invariants; do not optimize unrelated code.","expected_invariants":["Separates whole-process RSS from Go runtime-managed memory instead of equating either with the live Go heap","Accounts for the unexplained footprint using evidence for cgo, mmap, the binary, thread stacks, and operating-system-held memory","Derives GOMEMLIMIT below the container hard limit from measured non-Go headroom and burst risk rather than a universal percentage","Distinguishes live heap ownership, cumulative allocation churn, runtime-reserved memory, releasable pages, and released pages","Uses heap or allocation profiles to identify retaining owners or churn before adding pooling or changing GC policy","Checks heap live versus goal, GC CPU and assist time, forced cycles, and the GC limiter signal when evaluating the limit","Measures achieved throughput, deadline failures, CPU, and tail latency because a low soft limit can trade memory for near-continuous collection","Treats periodic FreeOSMemory as a workload-changing intervention that requires evidence rather than a substitute for finding the memory owner"],"forbidden_outcomes":["Sets GOMEMLIMIT equal to the container limit and claims this prevents process OOMs","Calls rising RSS a Go heap leak solely from a process graph","Uses a forced-GC loop as the primary repair without accounting for external or retained memory"],"graders":[{"id":"memory-budget-contract","kind":"contains","required":["GOMEMLIMIT","cgo"],"weight":1}]},{"id":"quality-production-profiling-review","kind":"quality","split":"development","prompt":"Review a Go 1.26 production-profiling change. It blank-imports net/http/pprof and serves http.DefaultServeMux on the public application listener without authentication. During an overload, an incident job starts a 60-second CPU profile and execution trace on every pod while a continuous CPU profiler is active, sets block profile rate and mutex profile fraction to sample every event without restoring them, requests a forced-GC heap profile, and compares the result with a previous build under different GOMAXPROCS and traffic. It uploads cmdline, goroutine, heap, and trace artifacts to a public ticket. Define access, cohort, interference, sampling, comparison, and artifact invariants.","expected_invariants":["Uses an explicit private administrative listener and mux with non-bypassable authentication authorization network restriction and collection audit","Inventories the pprof surface including command line symbols profiles and trace rather than exposing the default mux publicly","Selects the cheapest profile that can answer the hypothesis and measures its overhead under representative load","Coordinates manual HTTP and continuous profilers through one owner because only one CPU profile can be active","Staggers bounded collection across a small representative cohort instead of profiling every overloaded replica together","Records and restores process-wide block and mutex sampling settings and does not leave maximal sampling enabled","Labels forced garbage collection and overlapping diagnostics as workload-changing interventions","Binds profiles to build identity source Go version architecture GOMAXPROCS runtime settings deployment cohort interval workload throughput errors and sampling configuration","Protects profile artifacts as potentially sensitive operational data with access encryption retention and deletion controls"],"forbidden_outcomes":["Exposes net/http/pprof unauthenticated on the public listener","Starts CPU profiling independently while another CPU profiler owns the process","Treats forced-GC or differently sampled profiles from different builds and traffic as a clean before-after comparison","Publishes command line stacks labels or retained-object evidence to an unrestricted ticket"],"graders":[{"id":"production-profiling-contract","kind":"contains","required":["pprof","cohort"],"weight":1}]}]}