Update to ABMeter

{
  "description": "Analyze an ABMeter experiment's results, recommend a decision, and guide the user through concluding it on the website. Use when the user wants to stop an experiment or pick a winner.",
  "included_files": [],
  "name": "conclude-experiment",
  "skill_md_contents": "---\nname: conclude-experiment\ndescription: \"Analyze an ABMeter experiment's results, recommend a decision, and guide the user through concluding it on the website. Use when the user wants to stop an experiment or pick a winner.\"\n---\n\nReplace `<experiment-slug>` below with the slug of the experiment the user named. If they named none, find it with `search_experiments` and ask which one.\n\nI want to conclude the experiment '<experiment-slug>'.\n\n## Your Workflow (State-Aware)\n\nFirst, use `search_experiments` to find '<experiment-slug>' and check its current state. Note its `space_slug`: pass it as `space_slug` to every tool call below.\n\n### If RUNNING (most common path):\n\n**1. Analyze effects FIRST -- before any destructive action**\n- Use `search_experiments` to get experiment details (hypothesis, primary metric, audiences, variants)\n- Use `evaluate_experiment_effects` with the primary metric (and guardrail metrics if set) to compute statistical results\n- Interpret results: p-values, confidence intervals, effect sizes, sample sizes\n- This analysis happens while the experiment is still live -- no user impact yet\n\n**2. Formulate a recommendation**\nBased on the statistical analysis, recommend one of:\n- **Apply Winner** (specific audience): statistically significant positive effect (p < 0.05, CI excludes 0, meaningful effect size)\n- **Keep Original Values**: control performs best or no significant difference -- results are conclusive but treatment didn't win\n- **Reset to Draft**: data quality issues (too few users, implementation bugs, no events tracked)\n- **Don't stop yet**: results are inconclusive and more data could help -- estimate how much longer based on current traffic rate\nAlways explain the reasoning clearly.\n\n**3. Present recommendation to user BEFORE stopping**\n- The recommended decision with rationale\n- What will happen when that decision is applied (e.g., \"Parameter `button_size` default will change from `medium` to `large` for all users\")\n- Suggested decision notes text\n- **CRITICAL: Get user agreement before proceeding to destructive actions**\n- Stopping is destructive: it removes the experiment from assignment_config, reverting all users to control\n\n**4. Guide through UI actions (stop + decide)**\nAll state changes go through the Web UI for safety and auditability:\n- Use `navigate_browser` with `resource_type: \"experiment\"` and `resource_slug: \"<experiment-slug>\"` to open the experiment page\n- Instruct user to click **\"Stop\"** button and confirm -- wait for their confirmation\n- After user confirms stop, use `search_experiments` to verify state is `stopped`\n- Navigate to the decision page with the recommendation pre-populated:\n  Use `navigate_browser` with `resource_type: \"experiment_decision\"`, `resource_slug: \"<experiment-slug>\"`, and `query_params` containing the recommendation as a JSON string.\n  Example query_params:\n  ```json\n  {\"decision\": \"apply_winner\", \"winning_audience_slug\": \"<slug from audiences array>\", \"decision_notes\": \"<your statistical summary>\"}\n  ```\n- The decision modal opens automatically with the recommendation pre-selected, winning variant chosen, and decision notes filled in\n- Tell the user: \"The decision modal is open with my recommendation pre-selected. Review it and click **Confirm Decision**, or change the selection if you disagree.\"\n\n**5. Verify the outcome**\n- Use `search_experiments` to verify the experiment transitioned to `archived` state (or `init` if reset)\n\n### If STOPPED (already stopped, need decision):\n\nSkip the stopping steps. Go directly to:\n1. Analyze effects (same as above -- `evaluate_experiment_effects` works on stopped experiments too)\n2. Formulate recommendation\n3. Present recommendation\n4. Navigate to decision page with pre-populated recommendation using `navigate_browser` with `resource_type: \"experiment_decision\"` and `query_params` (use `winning_audience_slug` with a slug from the experiment's `audiences` array)\n5. User confirms or overrides in the pre-populated modal\n6. Verify outcome\n\n### If ARCHIVED (already concluded):\n\nInform the user the experiment has already been concluded. Use `search_experiments` to show the existing `decision_data` summary.\n\n### If INIT (not started yet):\n\nInform the user the experiment hasn't been started yet. Suggest using the `design-experiment` skill if they need to launch it.\n\n## CRITICAL RULES\n\n- **Experiments MUST be stopped manually by the user through the Web UI.** You CANNOT and MUST NOT stop/transition experiments programmatically.\n- **Analysis before action**: Form your conclusion while the experiment is still running. The user should know what they're going to decide before touching anything destructive.\n- **DO NOT use curl, APIs, or any tools to change experiment state.** All state changes happen through user clicks in the Web UI.\n- Use simple language focused on helping the user make a well-informed decision.\n"
}