Update to Keel

{
  "description": "Run a backtest on an existing strategy and read it: confirm the strategy, run on the strategy's default window unless the user named one, watch to terminal, present the run's view, and interpret the mechanism behind the result. Use when the user says \"run a backtest\", \"test this\", \"how does it perform\", \"what's the Sharpe\", or names a strategy and asks about performance. Not for robustness checks (overfit-check) or for editing the strategy.\n",
  "included_files": [],
  "name": "backtest-and-analyze",
  "skill_md_contents": "---\nname: backtest-and-analyze\ndescription: |\n  Run a backtest on an existing strategy and read it: confirm the strategy, run on the strategy's default window unless the user named one, watch to terminal, present the run's view, and interpret the mechanism behind the result. Use when the user says \"run a backtest\", \"test this\", \"how does it perform\", \"what's the Sharpe\", or names a strategy and asks about performance. Not for robustness checks (overfit-check) or for editing the strategy.\nreferences:\n  - step: 2\n    topic: \"tool_usage\"\n    why: \"backtest mechanics: one continuous simulation, half-open window, warm-up; never N short runs as a substitute for post-processing\"\n  - step: 2\n    topic: \"platform_operations\"\n    why: \"the backtest window rule (the default window per timeframe, warm-up, the data floor) and per-asset data coverage\"\n  - step: 2\n    topic: \"backtest_costs\"\n    why: \"backtest cost assumptions: the default fee and slippage model every run applies\"\n  - step: 4\n    topic: \"trading_domain\"\n    why: \"timeframe facts (too few trades on a short bar), trend vs mean-reversion behaviour, the complexity ladder\"\n  - step: 4\n    topic: \"mistakes\"\n    why: \"structural mistakes behind a zero-trade or wrong-direction result\"\ntools:\n  - keel_strategy_search\n  - keel_strategy_notes_read\n  - keel_backtest_run\n  - keel_backtest_watch\n  - keel_backtest_summarize\n  - keel_backtest_compare\n  - keel_strategy_history\n  - keel_strategy_notes_add\n---\n\n# References\n\nReference topics this workflow cites (keel_help topics):\n\n- Step 2 — `tool_usage` — backtest mechanics: one continuous simulation, half-open window, warm-up; never N short runs as a substitute for post-processing\n- Step 2 — `platform_operations` — the backtest window rule (the default window per timeframe, warm-up, the data floor) and per-asset data coverage\n- Step 2 — `backtest_costs` — backtest cost assumptions: the default fee and slippage model every run applies\n- Step 4 — `trading_domain` — timeframe facts (too few trades on a short bar), trend vs mean-reversion behaviour, the complexity ladder\n- Step 4 — `mistakes` — structural mistakes behind a zero-trade or wrong-direction result\n\n# Method\n\n## Step 1 — Confirm inputs and read prior context\n\n- A `strategy_id` from the user, else `keel_strategy_search(limit=5)` and\n  confirm which one.\n- `keel_strategy_notes_read(strategy_id=<id>)` — prior runs and notes may already\n  answer the question or say what to look at.\n- Window: omit `start_date` / `end_date` unless the user named a period; the\n  tool applies the strategy clock's default window and the receipt names the\n  window it used.\n- More than one run planned (a grid, several windows)? Each run uses one\n  backtest of the plan's weekly allowance; say how many runs the batch needs\n  before starting, and let the user scope it.\n\n## Step 2 — Run and wait\n\n`keel_backtest_run(strategy_id=<id>, ...)`. The receipt carries `run_id`,\n`hero_url`, and usually `summary_metrics`. If the run is still active, or\nmetrics are absent, `keel_backtest_watch(backtest_id=<run_id>)` — not an ad hoc\npolling loop. Once terminal, show the run with `keel_backtest_summarize` if it\nis the run you present; for several runs, `keel_backtest_compare` once with\nevery id. Backtests run the platform's current version; pin a historical one\nwith `commit_id=...` (from `keel_strategy_history`) only when that is what the user\nwants. A run may not finish in the turn: say it started and how to read it;\nnever promise to report back on your own.\n\n## Step 3 — Read the result\n\nThe `view` is the metrics table — where the host draws it the user already\nsees it; elsewhere its markdown stands in. Then, in prose: the date range\n(always — comparisons across ranges are meaningless), the top-3 / bottom-3\ncontributors (a Sharpe carried by one asset is a different result from one\nspread across thirty), best and worst month. `Sharpe > 3.0` or `Total Return >\n500%` on a long window is unusual for a real edge; say so plainly —\n`overfit-check` is the robustness check for a result like that.\n\n## Step 4 — Reason about it\n\nThree bullets: did the thesis work as designed or for another reason; where\ndid it underperform and is that consistent with the strategy type (trend\nbleeds in chop); what would a principled next iteration change — a mechanism\nfix with a reason independent of the backtest, never a change designed to\ndodge one bad period. Diagnose the mechanism, not the outcome: \"mean-reversion\nshorts in a parabolic breakout with no trend filter\" is a mechanism; \"shorts\nlost money in Nov 2024\" is an outcome. Report the result first, then the\ncaveats.\n\n## Step 5 — Save a note when asked\n\nWhen the user asks to save or remember the result,\n`keel_strategy_notes_add(strategy_id=<id>, note=<one paragraph>)`: Sharpe,\nreturn, max DD, window, one-line interpretation. End the reply with the\n`hero_url` on its own line.\n\n# Decision points\n\n- One run → summarize; several → compare once. Never both for the same run.\n- Plan-limit refusals (`quota_exhausted`, `quota_cap_reached`,\n  `plan_feature_unavailable`) carry the numbers: report the limit and the\n  reset. The call is not retried; re-authenticating adds no capacity.\n- Simulation framing (daily reset, withdrawals, \"add $X a week\") is not a\n  reason for N short backtests: run one over the full window and decompose\n  from trades and equity.\n\n# Output shape\n\n1. One sentence: strategy, Sharpe, return, max DD, window.\n2. The run's `view` — explained, not repeated, where the host draws it.\n3. Top-3 / bottom-3 contributors.\n4. Three bullets of \"what this means\".\n5. `hero_url` on its own line.\n6. Where a follow-up is natural, one line naming it — a robustness check\n   (`overfit-check`) for an unusually strong result, a mechanism fix\n   (`strategy-fork-and-iterate`) for a clear weakness — for the user to pick.\n\n# When NOT to use\n\n- Robustness / out-of-sample → `overfit-check`.\n- Comparing runs → `keel_backtest_compare`.\n- Creating or editing the strategy first → `strategy-creation` /\n  `strategy-fork-and-iterate`.\n\n# Test prompts\n\n1. \"Run a backtest on str_K9p2Lz from <start> to today.\"\n2. \"What's the Sharpe of my funding carry strategy?\"\n3. \"Test the strategy I just created.\"\n"
}