skills/go-data-consistency/evals.json

{"schema_version":2,"skill":"go-data-consistency","cases":[{"id":"route-write-skew","kind":"routing","split":"development","prompt":"Two Go SQL transactions pass a limit check and commit a state that violates a cross-row invariant.","should_activate":true,"reason":"Isolation and durable invariant dominate."},{"id":"avoid-saga","kind":"routing","split":"development","prompt":"Coordinate compensation across three services after a partial workflow failure.","should_activate":false,"reason":"Cross-service recovery belongs to go-distributed-coordination.","confuses_with":["go-distributed-coordination"]},{"id":"quality-commit-ambiguity","kind":"quality","split":"development","prompt":"Fix the fixture so a lost commit response never becomes a definite rollback or a blind retry.","fixture":"evaluations/fixtures/commit-ambiguity","expected_invariants":["Treats outcome as potentially committed","Uses stable identity or authoritative reconciliation"],"forbidden_outcomes":["Blindly retries with a new identity"],"graders":[{"id":"ambiguous-commit","kind":"go-test","target":"./...","weight":1}]},{"id":"quality-cache-order","kind":"quality","split":"development","prompt":"Two invalidation events arrive out of order and resurrect stale account state.","expected_invariants":["Defines source-of-truth version ordering","Prevents older value replacing newer"],"forbidden_outcomes":["Assumes best-effort invalidation is ordered"],"graders":[{"id":"cache-version","kind":"contains","required":["version","stale"],"weight":1}]},{"id":"quality-versioned-cache","kind":"quality","split":"development","prompt":"Repair the fixture without changing its exported API. Events for a key can be duplicated and delivered out of order; Version increases at the source. After version N is accepted, lower or equal versions must not change visible state. A deletion must block older updates from resurrecting data, while a strictly newer update may recreate it.","fixture":"evaluations/fixtures/versioned-cache","expected_invariants":["Lower and equal versions cannot replace accepted state","A deletion retains enough ordering state to reject older updates","A strictly newer update can recreate a deleted key"],"forbidden_outcomes":["Deletes all ordering state with the cached value"],"graders":[{"id":"versioned-cache","kind":"go-test","target":"-race ./...","weight":1}]},{"id":"quality-online-schema-cutover","kind":"quality","split":"development","prompt":"Review an online Go/PostgreSQL migration that adds a required normalized_state column and unique lookup index, deploys old and new writers concurrently, backfills with LIMIT/OFFSET and unconditional updates, switches reads when the last batch commits, and immediately drops the legacy column. Production writes continue throughout and rollback may run the old binary. Report only correctness and availability invariants; do not prescribe engine features without their failure conditions.","expected_invariants":["Uses an additive expand phase compatible with every concurrently deployed reader and writer","Backfills in bounded restartable stable-key batches rather than offset pagination over changing rows","Uses conditional ownership or version checks so backfill cannot overwrite a newer application write","Proves authoritative row completeness and validates constraints before switching reads","Accounts for PostgreSQL lock, transaction, and failed-index behavior when choosing NOT VALID validation or concurrent index construction","Delays contract until old binaries, queued jobs, consumers, and rollback paths cannot access the legacy representation"],"forbidden_outcomes":["Treats completion of the final scheduled batch as proof every row was migrated","Assumes CREATE INDEX CONCURRENTLY is failure-free or valid after an interrupted build","Drops the legacy column while an old binary or rollback can still write it"],"graders":[{"id":"online-migration-review","kind":"contains","required":["backfill","rollback"],"weight":1}]},{"id":"quality-replica-read-authority-review","kind":"quality","split":"development","prompt":"Review a Go/PostgreSQL reservation API for consistency. POST commits a reservation on the primary and returns 201. The following GET always uses an asynchronous hot standby; when the row is absent, the client reports the POST rolled back and retries the same logical reservation under a new operation ID. The service pins that user to the standby for two seconds and claims `synchronous_commit=on` makes the row visible there. During failover a lagging standby is promoted. State the read-authority, retry, and failover invariants without assuming another database has PostgreSQL semantics.","expected_invariants":["Explains that primary commit success does not prove an asynchronous standby has replayed the transaction","Treats a missing replica row as a stale observation rather than evidence that the POST rolled back","Keeps one stable operation identity so retry or reconciliation cannot create a second reservation","Routes correctness-dependent reads to the authority or carries a commit token and waits for replay with a bounded authoritative fallback","Rejects a fixed two-second pin as proof of read-your-writes because it observes no replay boundary","Explains that PostgreSQL synchronous_commit on does not by itself make an arbitrary standby query current","Qualifies remote_apply as visibility on the selected current synchronous standbys in the documented PostgreSQL case with latency and availability costs","Handles promotion under asynchronous replication as possible loss of acknowledged writes and reconciles against the new fenced authority","Binds replay evidence to the relevant cluster or timeline generation and monitors the actual serving replica's replay position"],"forbidden_outcomes":["Retries the logical reservation under a fresh identity after a replica miss","Treats cache state or elapsed time as authoritative replay evidence","Claims synchronous commit universally guarantees replica visibility or failover retention"],"graders":[{"id":"replica-read-contract","kind":"contains","required":["replica","operation"],"weight":1}]},{"id":"quality-pagination-consistency-review","kind":"quality","split":"development","prompt":"Review a multi-tenant Go/PostgreSQL audit-log endpoint. It orders rows only by created_at DESC and returns a base64 cursor containing that timestamp. The next request can change tenant_id, filters, and sort direction while reusing the cursor; every page reads any asynchronous replica in Read Committed. Concurrent inserts can share timestamps, existing rows can have created_at corrected, and a lagging replica can be promoted between pages. The API promises both that users see a live feed and that the export is a stable snapshot with no duplicates or omissions. State the order, cursor-authority, mutation, snapshot, replica, and failover invariants and resolve the contradictory contract.","expected_invariants":["Requires a deterministic total order with a unique tie-breaker and an exact lexicographic continuation predicate over every sort component","Specifies direction, null, and collation semantics and rejects a timestamp-only boundary for equal timestamps","Binds the cursor to tenant or authority scope, normalized filters, sort policy, API version, boundary, data version when used, topology generation, and expiry","Authenticates the cursor when tampering matters and explains that base64 supplies neither integrity nor authorization","Reauthorizes every request rather than treating a previously valid cursor as an access grant","Chooses either documented live traversal or a stable snapshot or watermark contract instead of promising both incompatible views","Explains that keyset pagination can still duplicate or skip rows whose sort key changes","Explains that separate Read Committed requests do not share one PostgreSQL snapshot","Requires the serving replica to satisfy the cursor's version or replay boundary, with an authoritative fallback or explicit stale-cursor result","Binds database positions to a comparable cluster or timeline generation and defines restart or rejection after failover","Uses a bounded page plus one and evaluates index support without claiming keyset is universally faster"],"forbidden_outcomes":["Claims cursor pagination is inherently stable under all concurrent writes","Treats base64 opacity or a signed cursor as sufficient tenant authorization","Continues an old replica position against an incomparable promoted history without detection"],"graders":[{"id":"pagination-consistency-contract","kind":"contains","required":["order","snapshot"],"weight":1}]},{"id":"quality-etcd-watch-resync-review","kind":"quality","split":"development","prompt":"Review a Go control-plane projection backed by etcd v3.6. Startup ranges a prefix, populates the live map, then creates a watch with no start revision. Every watch response advances LastRevision before its events are applied concurrently. A progress notification marks the service ready. On compaction, the code clears the live map and retries the same revision. After an etcd snapshot restore, it reuses the old numeric checkpoint. Policy updates can modify several keys in one transaction, and serving a mixed policy revision is unsafe. State the snapshot, watch, application, readiness, compaction, lineage, and resource invariants.","expected_invariants":["Builds a consistent prefix snapshot at a recorded source revision R","Starts the corresponding watch at R plus one to close the list-to-watch gap","Publishes all events from one source revision as one complete local transition","Preserves source revision order when later policy state depends on earlier revisions","Advances the applied checkpoint only after the complete revision is locally applied","Treats an etcd progress notification as delivery evidence through a revision rather than proof of local application","Derives readiness from the locally applied revision and an explicit staleness contract","Recovers a compacted checkpoint by taking a fresh snapshot instead of retrying the unavailable revision","Builds validates and catches up a private replacement generation before atomically replacing the live projection","Treats watch cancellation stream errors and decode failures as explicit degraded or resync states","Binds a checkpoint to range filters schema cluster identity and logical lineage","Resnapshots after restore and uses the documented revision-bump mechanism only as watcher notification evidence","Bounds snapshot memory event buffering reconnect rate resync time and stale serving"],"forbidden_outcomes":["Starts an unrevisioned watch after listing and claims no update can be missed","Clears the active projection before a replacement snapshot is complete","Treats a progress notification as proof all application callbacks finished","Compares an old checkpoint to a restored cluster by revision number alone"],"graders":[{"id":"etcd-watch-resync-review","kind":"contains","required":["revision","compaction"],"weight":1}]}]}