skills/go-message-processing/evals.json

{"schema_version":2,"skill":"go-message-processing","cases":[{"id":"route-outbox-redelivery","kind":"routing","split":"development","prompt":"Design a Go consumer and outbox relay for crashes, duplicate delivery, partition ordering, and poison messages.","should_activate":true,"reason":"Broker delivery and acknowledgement dominate."},{"id":"avoid-channel-pipeline","kind":"routing","split":"development","prompt":"An in-process Go channel pipeline deadlocks when its consumer exits.","should_activate":false,"reason":"In-process channel lifecycle belongs to go-concurrency-lifecycle.","confuses_with":["go-concurrency-lifecycle"]},{"id":"quality-effect-ack","kind":"quality","split":"development","prompt":"Fix the fixture so the durable idempotent effect commits before acknowledgement and redelivery is safe.","fixture":"evaluations/fixtures/message-redelivery","expected_invariants":["Commits durable effect before acknowledgement","Handles redelivery idempotently"],"forbidden_outcomes":["Acknowledges on enqueue to memory"],"graders":[{"id":"ack-order","kind":"go-test","target":"./...","weight":1}]},{"id":"quality-outbox","kind":"quality","split":"development","prompt":"Close the gap between database state and event publication.","expected_invariants":["Writes domain state and outbox atomically","Allows relay republish and consumer deduplication"],"forbidden_outcomes":["Assumes two independent commits are atomic"],"graders":[{"id":"outbox-atomicity","kind":"contains","required":["outbox","transaction"],"weight":1}]},{"id":"quality-rebalance-ownership-review","kind":"quality","split":"development","prompt":"Review a Go consumer-group worker for rebalance safety. Assignment generation 7 starts one goroutine per record but each job stores only the partition and offset. On revocation the callback commits maxFinished+1 asynchronously, ignores the result, and returns; workers keep running. Generation 8 receives the partition and starts replay. A generation-7 worker at offset 120 later completes an ambiguous billing call and updates the shared max, while offset 116 is still running. Cooperative assignment and static membership are enabled. State the ownership, progress, and effect invariants without choosing a client library.","expected_invariants":["Creates a fresh partition-scoped owner context and generation or epoch for each assignment","Stops admission for revoked partitions and cancels or drains their workers within the rebalance budget","Commits only through the highest contiguous completed offset rather than the largest observed completion","Checks offset commit success while the committing assignment is still valid and treats unknown or rejected commit as unresolved","Prevents late callbacks from a retired generation from updating a successor generation progress frontier","Explains that broker rejection of a stale generation offset commit does not fence an external billing effect already started","Makes the business effect converge under old-owner completion and new-owner replay through stable identity and durable idempotency or an authoritative fence","Treats cooperative assignment and static membership as churn controls rather than revocation safety","Defines replay from the last accepted offset as expected recovery rather than proof of duplicate-free delivery"],"forbidden_outcomes":["Commits maxFinished plus one while a lower offset is incomplete","Assumes returning from the revoke callback stops existing goroutines","Claims static membership or cooperative rebalance eliminates stale-owner effects"],"graders":[{"id":"rebalance-generation-contract","kind":"contains","required":["generation","contiguous"],"weight":1}]},{"id":"quality-cdc-snapshot-handoff-review","kind":"quality","split":"development","prompt":"Review a Go/PostgreSQL CDC bootstrap. It scans a large table in Read Committed key pages, records the current LSN only after the final page, and then starts logical decoding from that LSN. Snapshot rows and streamed changes are applied by separate workers with last-arrival-wins. Deletes remove both the value and all ordering metadata. The consumer acknowledges an LSN when work enters an in-memory queue, marks the replica ready when the scan ends, and drops the replication slot during disk pressure before creating a new slot at the current head. Failover can promote another timeline. State the snapshot, stream, durability, retention, and failover invariants without assuming arbitrary CDC products implement Debezium's protocol.","expected_invariants":["Uses a source-consistent snapshot aligned with a retained stream position so commits during the scan cannot fall into a gap","Rejects a Read Committed page scan followed by the latest LSN as a complete history","Resolves per-key snapshot and stream collisions using source position or a documented window so an older snapshot cannot overwrite a newer update","Retains delete ordering evidence or tombstones so an older snapshot or replay cannot resurrect deleted state","Advances or acknowledges source progress only after the corresponding sink effect is durable","Treats replication-slot crash replay as expected duplicate delivery requiring downstream idempotency","Marks serving cutover ready only after durable snapshot completion and stream catch-up through the declared handoff fence","Monitors and bounds retained WAL without dropping the only required history merely to relieve disk pressure","Binds checkpoints and positions to the source cluster or timeline generation publication filter and schema contract","Fences the old source and resnapshots when failover continuity cannot be proven"],"forbidden_outcomes":["Claims a scan completion timestamp can replace a source log position","Uses arrival order to resolve snapshot and stream conflicts","Treats slot advancement or removal as proof the sink is durable"],"graders":[{"id":"cdc-handoff-contract","kind":"contains","required":["snapshot","position"],"weight":1}]},{"id":"quality-poison-redrive-review","kind":"quality","split":"development","prompt":"Review a Go consumer whose queue sends a payment event to a dead-letter queue after three receives. Operators redrive the entire queue at maximum velocity after deploying a schema fix. The broker assigns new message IDs and enqueue times, redriven records interleave with new traffic, and deduplication entries expire before quarantine retention. Later events for the same account have already applied, queue payloads contain sensitive data, and canceling redrive leaves moved messages active. Define classification, evidence, ordering, identity, access, capacity, and reconciliation invariants.","expected_invariants":["Distinguishes transient systemic permanent business ambiguous and consumer-generation failures before quarantine","Preserves application identity fingerprint schema ordering key source position attempts failure class and immutable original payload evidence","Does not rely on broker message ID receive count or enqueue time as durable business identity","Extends deduplication or inbox protection through quarantine and replay or uses a stable business identity","Makes per-key sequence gaps and redrive interleaving explicit instead of claiming FIFO recovery","Revalidates under current schema authorization and policy without bypassing the normal consumer contract","Uses an approved bounded redrive generation with cohort velocity pause abort and capacity evidence","Protects sensitive payloads and reconciles moved deduplicated succeeded requarantined ambiguous and missing outcomes"],"forbidden_outcomes":["Treats dead-letter placement as business resolution","Replays at maximum speed solely to empty queue depth","Creates a new financial operation identity for an ambiguous original effect","Assumes canceling redrive removes messages already moved"],"graders":[{"id":"poison-redrive-review","kind":"contains","required":["identity","reconcil"],"weight":1}]}]}