skills/go-security-hardening/evals.json

{"schema_version":2,"skill":"go-security-hardening","cases":[{"id":"route-ssrf-command","kind":"routing","split":"development","prompt":"Review a Go endpoint that accepts URLs and command arguments from tenants and runs privileged work.","should_activate":true,"reason":"Untrusted data reaches network and execution authority."},{"id":"avoid-pci-scope","kind":"routing","split":"development","prompt":"Map cardholder-data scope and audit controls for a payment service.","should_activate":false,"reason":"Financial compliance boundaries belong to go-fintech-security-compliance.","confuses_with":["go-fintech-security-compliance"]},{"id":"quality-ssrf","kind":"quality","split":"development","prompt":"Harden an image fetcher against redirects, DNS rebinding, and private-network targets.","expected_invariants":["Constrains resolved destinations and redirects","Uses bounded transport and response size"],"forbidden_outcomes":["Validates only URL text once"],"graders":[{"id":"ssrf-path","kind":"contains","required":["redirect","resolved"],"weight":1}]},{"id":"quality-authz","kind":"quality","split":"development","prompt":"Fix a tenant handler that authenticates users but loads resources by arbitrary ID.","expected_invariants":["Binds subject tenant action and resource","Checks authoritative object after resolution"],"forbidden_outcomes":["Treats authentication as authorization"],"graders":[{"id":"authorization-boundary","kind":"contains","required":["tenant","resource"],"weight":1}]},{"id":"quality-dependency-integrity-review","kind":"quality","split":"development","prompt":"Review this Go release change for supply-chain integrity. CI now sets GOSUMDB=off, go.mod replaces a public authorization module with ../auth-fork, go:generate downloads and executes the latest generator, and the release job runs go mod verify plus govulncheck before publishing a binary SHA-256. The author claims these checks prove the dependency is trusted, the build is reproducible, and the binary is safe. Report only the guarantee gaps and the smallest evidence or control needed for each; preserve a correct private-module privacy boundary.","expected_invariants":["Distinguishes public module byte authentication through go.sum and the checksum database from source or maintainer trust","Explains that go mod verify detects modification of downloaded cache content rather than reviewing source trust or vulnerabilities","Flags GOSUMDB=off as removing first-use public checksum verification while preserving scoped private-module exclusions","Treats a local replace as unversioned checkout input outside public module checksum authentication","Includes downloaded generators and other non-module build inputs in pinning and provenance","Treats govulncheck as evidence about known reachable vulnerabilities under its analyzed configuration rather than proof of absence","Treats the published binary checksum as artifact identity and requires trustworthy build provenance for source-to-artifact claims"],"forbidden_outcomes":["Claims a matching checksum proves a dependency or binary is trustworthy","Claims go mod verify is a vulnerability scanner","Disables private-module privacy controls merely to force public checksum lookup"],"graders":[{"id":"dependency-integrity-contract","kind":"contains","required":["checksum","replace"],"weight":1}]},{"id":"quality-trusted-proxy-identity-review","kind":"quality","split":"development","prompt":"Review a Go tenant API behind two paths. Internet traffic reaches a CDN and then a Go ReverseProxy using Director; it preserves inbound X-Forwarded-For and X-Client-Cert, appends its peer address, and forwards both to the backend. A maintenance load balancer can also reach the backend directly. The backend trusts the leftmost forwarded address for security rate limits and authorizes the tenant named by X-Client-Cert whenever the immediate source is in a private subnet. A shared response cache omits tenant and certificate identity from its key. State the transport, header, identity, and cache invariants without assuming private addressing authenticates a proxy.","expected_invariants":["Authenticates the immediate trusted proxy path or makes the backend unreachable from untrusted and alternate paths before accepting proxy metadata","Requires the first trusted ingress to remove attacker-supplied forwarded and client-certificate fields and reconstruct only evidence it owns","Defines the ordered multi-proxy chain and validates the trusted suffix or equivalent ownership before selecting a client element","Rejects a fixed leftmost-address rule because an attacker can prepend a value when the ingress preserves the inbound chain","Accepts client-certificate metadata only from a proxy that actually validated mTLS and protects the proxy-to-origin hop","Separates the maintenance path or makes it ignore proxy-only identity so direct access fails closed","Creates one structured authenticated identity at admission instead of repeatedly authorizing from raw headers","Partitions or disables shared cache reuse when authorization depends on certificate or tenant identity","Prefers ReverseProxy Rewrite sanitization and conditional reconstruction over Director's preserved inbound X-Forwarded fields"],"forbidden_outcomes":["Treats a private source address as sufficient proof that every forwarded value is authentic","Authorizes from any client-supplied certificate header","Selects the leftmost or rightmost forwarded element without a documented authenticated hop contract"],"graders":[{"id":"trusted-proxy-contract","kind":"contains","required":["proxy","header"],"weight":1}]},{"id":"quality-envelope-key-rotation-review","kind":"quality","split":"development","prompt":"Review a Go service that encrypts tenant documents with AES-GCM. Each row stores ciphertext and a wrapped DEK but no envelope, algorithm, KEK, or associated-data version. The row's client-supplied kms_uri selects which KMS key decrypts it. Rotation immediately changes the KEK alias, rewrites wrapped DEKs in parallel with unconditional updates, and destroys the old version when the scan reaches its last page; rollback binaries know only the old alias. A retry after a KMS timeout may replace a newer wrapped DEK, and operators claim KEK rewrap also repairs a leaked DEK and proves backups are cryptographically erased. State the envelope, rotation, compromise, cutover, and erasure invariants for Go 1.24-1.26 without designing a new cipher.","expected_invariants":["Uses a maintained AEAD or managed envelope construction and rejects attacker-selected arbitrary KMS key or endpoint authority","Persists an authenticated envelope format with algorithm, KEK version, wrapped DEK, nonce when exposed, ciphertext, and associated-data schema needed for deterministic decryption","Binds ciphertext to stable tenant, object, field-purpose, or schema context through canonical associated data without treating associated data as secret","Requires nonce uniqueness for each key and accounts for the Go random-nonce GCM per-key message limit rather than using an unpersisted counter","Distinguishes routine KEK rewrap from DEK replacement and full algorithm or format migration","Explains that rewrapping a leaked or overused DEK preserves the compromised data key and may require data re-encryption","Rolls out new-key decrypt capability before switching new encryption to that primary and keeps rollback and recovery readers compatible","Migrates by stable record identity and source/target versions with conditional ownership so retries cannot overwrite a newer envelope","Retains the old decryptable version across ambiguous KMS or database outcomes until the new envelope is durable and verified","Retires old keys only after evidence covers live data, replicas, queues, backups, recovery, and rollback paths rather than scan completion alone","Qualifies cryptographic erasure on absence of usable KEK or plaintext-DEK copies and does not claim Go garbage collection zeroizes key bytes"],"forbidden_outcomes":["Claims every key rotation only needs DEK rewrap regardless of DEK compromise or algorithm migration","Destroys the old KEK when the migration scheduler reaches its final page without decryptability evidence","Treats base ciphertext metadata or a client-provided KMS URI as trusted key authority"],"graders":[{"id":"envelope-rotation-contract","kind":"contains","required":["DEK","version"],"weight":1}]},{"id":"quality-external-process-lifecycle-review","kind":"quality","split":"development","prompt":"Review a Go document-conversion worker. A tenant chooses format and filename; the worker prepends the upload directory to PATH, runs sh -c with a concatenated command, inherits the service environment, and captures CombinedOutput. A two-second CommandContext timeout kills the shell. The converter forks a helper that can retain stdout, write the destination later, and survive the shell; the worker treats every timeout as no effect and retries under a fresh job ID. State the executable, argument, authority, resource, descendant, output, and replay invariants for Linux and Windows without selecting a sandbox product.","expected_invariants":["Selects an allowlisted executable identity through an explicit trusted path or validated lookup and does not suppress ErrDot to run a tenant-directory binary","Passes untrusted values as structured arguments and avoids a shell unless its separate grammar and quoting contract are justified and platform-specific","Constructs a minimal child environment and deliberately owns working directory standard streams and inherited files instead of delegating the service environment","Explains that Cmd.Dir and os/exec are not a filesystem network credential or kernel sandbox","Bounds input stdout stderr generated files time CPU memory file count and concurrency outside an unbounded CombinedOutput allocation","Keeps secrets out of process-list arguments inherited environment captured diagnostics and logs","Calls Wait exactly once after Start and distinguishes start exit cancellation deadline I/O WaitDelay and policy failures","Defines graceful and forced cancellation bounds appropriate to the helper rather than assuming a context proves shutdown","Explains that CommandContext defaults to killing the immediate process and does not establish portable descendant-tree termination","Uses a platform-specific process group job object cgroup container or cooperative supervisor only after defining the owned descendant set and preventing escape or caller-group termination","Uses WaitDelay to bound selected child and pipe waits without claiming pipe closure proves descendant termination","Publishes output atomically under a stable operation identity and reconciles a timeout because a child or external effect may have completed","Tests hostile path and arguments secret inheritance excessive output retained pipes descendant escape partial output and timeout replay on every supported OS"],"forbidden_outcomes":["Keeps sh -c and treats generic string escaping as a portable command-injection fix","Claims CommandContext kills every descendant or undoes its file and network effects","Treats Cmd.Dir or an empty environment as a complete sandbox","Retries every timeout with a new identity and overwrites whatever the first attempt later publishes"],"graders":[{"id":"external-process-boundary","kind":"contains","required":["process","WaitDelay"],"weight":1}]}]}