← CavemanCONTENT HISTORYWHAT CHANGED · RULE-BASED ANALYSIS
Update to Caveman
Snapshot Sep 30, 2026 · 23:17 UTC · version 2.7.0
Collection source: not recorded for this historical snapshot.
First saved snapshot
No earlier snapshot is available to establish a change.
Compare saved observations
Download comparison JSONFull technical diff · 0 changed fields
Full snapshot data
{
"name": "caveman-discover",
"description": "Use when the user asks to discover or label LLM workflows in a repository so Caveman can attribute spend by workflow.",
"included_files": [
{
"relative_path": "agents/openai.yaml",
"size_in_bytes": 237
}
],
"skill_md_contents": "---\nname: caveman-discover\ndescription: Use when the user asks to discover or label LLM workflows in a repository so Caveman can attribute spend by workflow.\n---\n## Host capability adaptation\n\n1. **Native host capability:** when ChatGPT/Codex exposes a native tool or connected source that satisfies this task, use it.\n2. **Original external runtime:** when the upstream runtime named by this skill is actually available, use it as documented below.\n3. **Instruction-only fallback:** when neither is available, perform only the reasoning/instruction portion that remains valid, state the limitation, and never fabricate tool output, successful execution, or persisted state.\n\nUse the original external runtime only when it is actually available; otherwise provide the valid instruction-based fallback and clearly disclose that execution was not performed.\n\n\nYou are labeling this repository's LLM workflows for Caveman Cloud. A\n*workflow* is a job the code performs — \"answer a support ticket\", \"build the\nnightly digest\", \"run the eval suite\" — not a technology. Every gateway\nrequest can carry a workflow label; unlabeled traffic all lands in one\n`unlabeled-workflow` bucket. Your job: find the workflows, name them well,\nwire the labels, and verify nothing broke.\n\nThis changes code, so it goes through the user's normal review: **propose the\ntable first, apply after the user agrees.** Re-running on an already-labeled\nrepo must change nothing (idempotent).\n\nThis skill is operator-invoked. An `unlabeled-traffic` Cave Plan observation is\nreview-only and does not create an advisory file, proposal, or Draft PR. Do not\ninfer that telemetry selected a callsite or authorized an edit. Independently\ninventory the repository, present the labeling table, and wait for the user's\napproval before changing code.\n\n## Step 1 — Inventory the workflows\n\nWalk the repo from its entry points, not from its imports:\n\n- HTTP/RPC handlers that call an LLM (directly or through layers)\n- Scheduled jobs: cron definitions, queue consumers, workers, GitHub Actions\n that invoke LLM code\n- CLI commands and scripts (`scripts/`, `bin/`, package.json scripts)\n- Eval / test harnesses that burn real tokens\n- Distinct agents or chains inside a framework (each LangGraph graph, each\n crew, each agent definition is usually its own workflow)\n\nOne workflow = one job a human would name. Ten callsites inside the same\nrequest handler are one workflow; one shared `llm.ts` helper used by three\njobs is three workflows (label at the callers, never the shared helper).\n\n## Step 2 — Name them\n\nSlug grammar (the gateway enforces this): lowercase `[a-z0-9_-]`, 1–96 chars.\nName the job, not the tech:\n\n- Good: `support-reply`, `nightly-digest`, `pr-review`, `eval-suite`,\n `onboarding-email`\n- Bad: `openai-calls` (tech), `main` (says nothing), `SupportReply` (invalid),\n `johns-test-3` (won't age)\n\nNames are forever-ish — renaming later splits the spend history. When a job's\npurpose isn't clear from the code, derive the slug from the file name and mark\nit `review` in the table rather than inventing a purpose.\n\n## Step 3 — Propose, then apply\n\nPresent this table and ask to proceed:\n\n```\n| workflow | job | where | how it gets labeled |\n|---|---|---|---|\n| support-reply | answers inbound tickets | src/bot/reply.ts:41 | defaultHeaders on the reply client |\n| nightly-digest | 02:00 summary job | jobs/digest.ts:12 | header on the digest client |\n| eval-suite (review) | scripts/eval.ts:8 — purpose inferred from filename | scripts/eval.ts:8 | env override at invocation |\n```\n\nThen wire each label with the lightest mechanism available at that callsite:\n\n- **@caveman-ai/sdk / caveman_cloud SDK**: per-trace `workflow` option, or\n `defaultWorkflow` on the client a single-job service constructs.\n- **Raw provider SDKs** (OpenAI/Anthropic/LangChain/LiteLLM/Vercel): add\n `\"x-cave-workflow\": \"<slug>\"` to the same `defaultHeaders` /\n `default_headers` / `extra_headers` block that already carries\n `x-cave-api-key`. Shared client used by several jobs → pass the header per\n call (every SDK above accepts per-request header overrides), or give each\n job its own thin client.\n- **Wrapped coding agents** (`caveman wrap`): `--workflow <slug>` flag or\n `CAVE_WORKFLOW=<slug>` env at the invocation site (cron line, CI step).\n- **Raw HTTP**: add the `x-cave-workflow` header to the request.\n\nLabel the callers, keep the diff minimal, match the repo's style. If a\ncallsite is not routed through the Caveman gateway at all, don't label it —\nlist it under \"not wired\" in the report (labels only travel on gateway\ntraffic; wiring is the caveman-setup skill's job).\n\n## Step 4 — Verify\n\nRun whatever the repo already uses to exercise one labeled path (a test, a\ndev script, one curl). Then confirm: the request still succeeds (the gateway\nrejects an invalid label with 400 `cave_invalid_request_header` — fix the slug\nif so). Labeled spend appears on the dashboard at `/activity?tab=workflows` as\neach workflow next runs; jobs on a schedule show up when the schedule fires,\nand that's worth saying in the report rather than pretending they're live.\n\n## Step 5 — Report\n\n```\n## Workflows labeled\n\n| workflow | job | where |\n|---|---|---|\n| support-reply | answers inbound tickets | src/bot/reply.ts:41 |\n| nightly-digest | 02:00 summary job | jobs/digest.ts:12 |\n\nVerified: <the labeled path you actually exercised, and what you observed>\nLands at: <DASHBOARD>/activity?tab=workflows — each row appears as that workflow\nnext runs. Anything still unlabeled shows as `unlabeled-workflow`.\nNot wired (no gateway routing, so no label): <list or \"none\">\nMarked review: <slugs whose purpose was inferred from filenames, or \"none\">\n```\n\nIf you found no LLM entry points at all: say exactly that, and point at the\nsetup skill (`<docs origin>/docs/agent-setup.md`) instead of manufacturing a\ntable.\n"
}SHA-256: 2dfa676ef9dd2f20a3bed2d8863dee17d785ed4a096067c75ee9b8270b0cc4d3