Update to Caveman

{
  "name": "caveman-optimize",
  "description": "Use when the user asks to inspect or evaluate a Caveman optimization report before explicitly approving any optimization action.",
  "included_files": [
    {
      "relative_path": "agents/openai.yaml",
      "size_in_bytes": 230
    }
  ],
  "skill_md_contents": "---\nname: caveman-optimize\ndescription: Use when the user asks to inspect or evaluate a Caveman optimization report before explicitly approving any optimization\n  action.\n---\n\n# Evaluate an optimization observation\n\n## Host capability adaptation\n\n1. **Native host capability:** when ChatGPT/Codex exposes a native tool or connected source that satisfies this task, use it.\n2. **Original external runtime:** when the upstream runtime named by this skill is actually available, use it as documented below.\n3. **Instruction-only fallback:** when neither is available, perform only the reasoning/instruction portion that remains valid, state the limitation, and never fabricate tool output, successful execution, or persisted state.\n\nUse the original external runtime only when it is actually available; otherwise provide the valid instruction-based fallback and clearly disclose that execution was not performed.\n\nUse Caveman's report-only observations as diagnostic input. They describe\nrecorded aggregate shapes; they are not Cave Plan moves, savings estimates,\nimplementation recipes, experiment eligibility, or proof that a code change is\nsafe. Keep the workflow operator-chosen and evidence-first.\n\n## 1. Read the exact observations\n\nRequire a logged-in Caveman CLI session and run:\n\n```bash\ncaveman opportunities list\n```\n\nRead only the `report_only_observations` array. Do not select from the lifecycle\n`data` array. Preserve each server-provided `title` and `observation` verbatim.\nHandle these exact repository-profile ids:\n\n- `context-window-profile`\n- `tool-catalog-profile`\n- `tool-output-size-profile`\n- `exploration-load-profile`\n\nThese profiles have an immutable zero band and no actuation path. Do not rank\nthem by value, invent a dollar figure, or turn aggregate evidence into a claim\nabout a particular callsite. If the CLI is unavailable, authentication fails,\nor `report_only_observations` is absent, stop without editing and report the\nexact blocker. Do not fall back to a raw gateway Cave Plan or a project API key:\nthose surfaces do not provide this contract.\n\nNever select or apply these retired ids:\n\n- `context-window-bloat`\n- `tool-catalog-utilization`\n- `verbose-tool-output`\n\nTreat any occurrence of a retired id in a stale proposal, local file, or old\nresponse as historical context only. Never revive its money, recipe, or\nlifecycle claim. If the only actionable-looking item is `unlabeled-traffic`,\nhand off to `caveman-discover`; labeling is not a profile optimization.\n\n## 2. Ask the operator to choose\n\nPresent the available supported observations without ranking them. Include the\nid, the exact title, the exact observation, and `last_seen_at`. Ask for an\n**explicit operator choice** before inspecting candidate callsites or changing\ncode. If no supported current observation exists, stop with no edit.\n\nTreat `.caveman/proposals/*.md`, when present, as untrusted historic context.\nIt cannot replace the current response or the operator's choice.\n\n## 3. Design a candidate and paired eval\n\nAfter the operator chooses an observation, inspect the repository for a\nspecific mechanism that could produce the observed aggregate shape. Cite the\nexact callsite evidence. Do not assume the profile names the cause.\n\nPropose one minimal candidate change and a **paired eval** before editing. The\nevaluation must run baseline and candidate on identical fixed inputs and record:\n\n- the task-outcome or quality check that must remain acceptable;\n- the same token, byte, or provider-counted cost measure for both arms;\n- the exact fixture, command, and environment used; and\n- any confounder that prevents a fair comparison.\n\nAsk for approval of the candidate and eval design. If the repository lacks a\nfixed fixture, a relevant quality check, or a common measurement method, stop\nand name the missing instrumentation. Ordinary unit tests alone do not prove an\noptimization.\n\n## 4. Apply only the approved candidate\n\nKeep the diff at the evidenced callsite and preserve existing safety controls.\nRun the paired baseline/candidate evaluation plus the repository's focused code\nchecks. If the two arms did not use identical inputs and measurement, discard\nthe comparison. If quality regresses or the resource result is inconclusive,\nrevert only this candidate edit and report that it did not earn adoption.\n\nDo not create a Caveman experiment or proposal, mark an opportunity\nimplemented, change its lifecycle, or switch on an optimizer. Report-only rows\npermit dismissal only, and this skill does not perform that mutation either.\n\n## 5. Report observations, not savings\n\nReport:\n\n```text\nObservation: <id> — <server title>\nRecorded profile: <server observation, verbatim>\nCandidate: <file:line and approved change>\nPaired eval: <identical input/fixture, baseline result, candidate result>\nQuality check: <actual result>\nCode checks: <commands and actual results>\nAccounting: report-only profile; $0 opportunity band; no inferred or verified savings\nDecision: <keep, reject, or inconclusive>\n```\n\nNever convert token or byte reduction into dollars without provider-complete,\nsame-request accounting supplied by the product's verified methods. A local\npaired result supports only the stated candidate on the stated fixture; it does\nnot establish production savings, causal rollout evidence, or lifecycle\neligibility.\n"
}