← OmnekyCONTENT HISTORY

Update to Omneky

Snapshot Oct 27, 2026 · 00:28 UTC · version 2.0.31

Collection source: downloaded plugin package.

WHAT CHANGED

Payment or plan references changed

Full description — Now says: launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use...

Observed in published text. Live prices and checkout terms have not been verified by this change.

Save Omneky and check future changes in Following.

Up to 3 plugins free · no card required.

All fields and original values.

Full description

Before

Oct 24, 2026 · 13:12 UTC

PAUSED launch) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), f...

After

Oct 27, 2026 · 00:28 UTC

launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (om...

Skill instructions

Before

Oct 24, 2026 · 13:12 UTC

PAUSED launch) or A/B test measurement (z-test, CI, (omneky-image-ads, omneky-product-video), or billing reads (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proporti...

After

Oct 27, 2026 · 00:28 UTC

launch live or paused as the user chooses) or A/B test measurement (z-test, CI, (omneky-image-ads, omneky-product-video, omneky-avatar-ugc), or credit balance (omneky-billing). Keywords: A/B test, ab test create, ab test measuremen...

Supporting files

Before

Oct 24, 2026 · 13:12 UTC

[]

After

Oct 27, 2026 · 00:28 UTC

[{"relative_path":"references/playbook.md","size_in_bytes":7149}]

Compare saved observations

Download comparison JSON

SKILL.md text changes

− removed · + added

--- before
+++ after
@@ -3,56 +3,47 @@
 description: >-
   Create or measure an Omneky paid-media A/B test. Use for A/B
   test creation (one hypothesis, one variable, precommitted
-  plan, PAUSED launch) or A/B test measurement (z-test, CI,
+  plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI,
   power, SRM, interim peek with no verdict, refuse a thin
   winner). Do not use for generic ROAS tables
   (omneky-analytics-and-catalogue), competitor analysis
   (omneky-competitor-analysis), first-run onboarding
   (omneky-onboarding), agency (omneky-agency), weekly marketing
   plans (omneky-marketing-plan), generating creatives
-  (omneky-image-ads, omneky-product-video), or billing reads
-  (omneky-billing). Keywords: A/B test, ab test create, ab test
-  measurement, which variant won, significance, two-proportion
-  z-test, MDE, SRM, interim peek, underpowered, commercially
-  meaningful, budget split.
+  (omneky-image-ads, omneky-product-video, omneky-avatar-ugc),
+  or credit balance (omneky-billing). Keywords: A/B test,
+  ab test create, ab test measurement, which variant won,
+  significance, two-proportion z-test, MDE, SRM, interim peek,
+  underpowered, commercially meaningful, budget split.
 ---
 
 # A/B tests
 
-Use the connected Omneky MCP server. Gate every named tool on this
-session's `tools/list`. If the tool for a requested step is missing,
-say so and point to the Omneky app. Do not invent a tool.
-
-This skill owns the **test plan** (create) and the **readout**
-(measure). Generic winner lists stay in
-`omneky-analytics-and-catalogue`. A one-off `PAUSED` launch without
-a test plan stays in `omneky-launch-and-connect`.
-
-Handoff fields:
-[handoff-contract](../_shared/references/handoff-contract.md)
-(including the A/B test plan fields).
-Formulas and worked examples (example data only):
-[ab-test-statistics](../_shared/references/ab-test-statistics.md).
-Sourced ids:
-[preconditions](../_shared/references/preconditions.md).
-Acceptance gates:
-[creative-quality-gates](../_shared/references/creative-quality-gates.md).
-Structured brief:
-[creative-brief](../_shared/references/creative-brief.md).
-Other intents:
-[capability-registry](../_shared/references/capability-registry.md).
-Write for:
-[marketer-persona](../_shared/references/marketer-persona.md).
+Use this when the user wants to set up an A/B test or read out which
+variant won. One job: own the test plan and the readout.
+
+Gate every tool on this session's `tools/list`. If a step's tool is
+missing, say so and point to the Omneky app.
+
+Plan fields: [handoff-contract](../_shared/references/handoff-contract.md).
+Formulas: [ab-test-statistics](../_shared/references/ab-test-statistics.md).
+Sourced ids: [preconditions](../_shared/references/preconditions.md).
+Acceptance gates: [creative-quality-gates](../_shared/references/creative-quality-gates.md).
+Brief: [creative-brief](../_shared/references/creative-brief.md).
+Other intents: [capability-registry](../_shared/references/capability-registry.md).
+Write for: [marketer-persona](../_shared/references/marketer-persona.md).
+Step detail: [playbook](references/playbook.md) (Hard rules, Create, Measure).
 
 ## Goal
 
-Create a one-variable `PAUSED` A/B test with a precommitted plan,
-or measure it with a two-proportion z-test and a marketer verdict.
+Create a one-variable test with a precommitted plan (live or paused is
+the user's choice in the host), or
+measure it with a two-proportion z-test and a marketer verdict.
 
 ## When to use
 
-- A/B test create, A/B test measurement, which variant won,
-  significance, stop rule, interim peek, SRM, MDE
+- Create an A/B test, measure one, "which variant won", significance,
+  stop rule, interim peek, SRM, MDE
 
 ## When not to use
 
@@ -64,171 +55,53 @@
 | Need | Check | If missing |
 |---|---|---|
 | Brand | `list_brands` | Ask which |
-| Channel | `get_channel_connection_status` | Connect first (`omneky-launch-and-connect`) |
-| Budget floor | `minimum_budget_for_objective` | Do not launch under the floor |
+| Channel | `list_connections` (`connector`) | Connect or managed |
+| Budget floor | `get_launch_requirements` | Do not launch under the floor |
 | Variants | prior creative handoff | Owning creative skill |
-| Baseline | `get_daily_metrics` with `date_range` | State an assumed `baseline_rate` |
-| Coverage (measure) | Metric tools for the window; empty is missing data | Do not declare a winner |
-| Metrics (measure) | `get_performance_breakdown` / `get_daily_metrics` | Narrow the window |
+| Baseline | `get_performance_summary` with `date_range` | State an assumed `baseline_rate` |
+| Coverage (measure) | `get_performance_summary` (`include_coverage=true`) | Do not declare a winner |
+| Metrics (measure) | `get_performance_breakdown` / `get_performance_summary` | Narrow the window |
 
 ## Flow
 
-Follow Create or Measure. **Invalid** is exempt from the
-horizon gate. The setup check (step 4) runs on every look,
-including pre-horizon peeks. If it fails, give the verdict
-Invalid immediately, Stop, and recommend pausing or fixing the
-test (a pause is a later yes via `omneky-launch-and-connect`;
-never mutate here). Only Winner, Not commercially
-meaningful, Inconclusive, and No detectable difference wait
-for `horizon`. A peek that passes the setup check is INTERIM
-with no statistical verdict. Refuse a winner when the sample
-is thin.
-
-## Hard rules
-
-- Stick to the chosen `brand_id`. Never silent-switch brands.
-- One hypothesis, **one** variable. Extra changes are a new test.
-- Default new launches to `PAUSED`. Live spend needs an explicit
-  yes in the same summary.
-- Call the budget-floor tool (`minimum_budget_for_objective`)
-  before any budget. Do not set a daily budget under that floor.
-  Split the approved total across variants; each arm stays at or
-  above the floor or you do not launch.
-- Paid creatives that precede the test use the owning creative
-  skill: Never invent a price.
-- Precommit and record the test plan: `primary_metric` (one),
-  `baseline_rate`, `mde`, `alpha` (default 0.05), `power`
-  (default 0.8), `n_per_arm` from the two-proportion formula
-  (show it), `randomization_unit` plus the ad-level-split caveat,
-  `attribution_window`, and `horizon` (end date or sample).
-- Measure: **Invalid** is exempt from the horizon gate. The
-  setup check runs on every look, including pre-horizon
-  peeks. If it fails, give the verdict Invalid immediately,
-  Stop, and recommend pausing or fixing (pause is a later yes via
-  `omneky-launch-and-connect`; never mutate here). Only
-  Winner, Not commercially meaningful, Inconclusive, and
-  No detectable difference wait for `horizon`. A peek that
-  passes setup is INTERIM. Early stop needs an explicit
-  acknowledgement and a Bonferroni or O'Brien-Fleming
-  correction (or say it is not supported).
-- Report effect size + CI, p-value, and achieved power.
-  Significance is not commercial meaning — compare to `mde` and
-  a CPA / ROAS $ impact.
-- SRM → Invalid only on a randomized planned split (platform
-  A/B / split-test, or an explicit planned allocation). Default
-  one-ad-group, one-ad-per-variant delivery is auction-driven:
-  do not run SRM as a gate; report imbalance as a caveat and
-  compare rates, not totals. Broken setup (near-zero delivery,
-  mid-test change, more than one variable) → Invalid. Low
-  counts → Insufficient data. Many variants → Bonferroni.
-- Scale / kill / iterate needs a later yes
-  (`omneky-launch-and-connect`).
-
-## Create
-
-1. Write the test plan with the user (one turn if they already
-   gave it):
-   - Hypothesis (what should win, and why)
-   - Single variable (creative, or headline, or audience — pick one)
-   - `primary_metric` (CTR or CVR — one only)
-   - `baseline_rate` from `get_daily_metrics` with `date_range`,
-     or a stated assumption
-   - `mde` (absolute rate)
-   - `alpha` 0.05 and `power` 0.8 unless they set others
-   - `n_per_arm` from the two-proportion formula (show it;
-     [ab-test-statistics](../_shared/references/ab-test-statistics.md))
-   - `randomization_unit` and the caveat that platform ad-level
-     splits are not true user randomization unless platform A/B
-     is used
-   - `attribution_window`
-   - `horizon` (end date or sample). This is the stop rule.
-2. Variants come from the structured brief (or a prior creative
-   handoff). Same brief except the one variable.
-3. Structure: one campaign, one ad group, one ad per variant
-   (same targeting and budget split). Use an existing campaign
-   only when they named it (`campaign_spec.id`).
-4. Preflight the channel with `omneky-launch-and-connect` rules:
-   connect status, budget floor, targeting pre-call, copy / CTA /
-   landing URL, credits. Default new launches to `PAUSED`.
-5. One summary: channel, tool, variants, budget split, floor,
-   the full test plan, `PAUSED`. Wait for one yes.
-6. Call the matching `launch_*`. Then write the test plan back
-   (`primary_metric`, `baseline_rate`, `mde`, `alpha`, `power`,
-   `n_per_arm`, `randomization_unit`, `attribution_window`,
-   `horizon`, campaign / ad ids).
-
-Do not go live unless that yes said to go live.
-
-## Measure
-
-1. Reuse the test plan. Missing ids: `get_campaigns` /
-   `get_ad_groups` (pass `campaign_id`) / `get_ads`. Do not guess.
-2. Empty metrics for the brand + window + channel mean no imported
-   data → stop. Empty is not zero.
-3. `get_performance_breakdown` (dimension ad or creative) and
-   `get_daily_metrics` for the window. `get_performance_movers`
-   when they asked what changed vs the prior period.
-4. Setup check on every look, including pre-horizon peeks.
-   **Invalid** only for a broken setup: near-zero delivery
-   on a variant, a variant changed mid-test, or more than
-   one variable differs. Default setup (one ad group, one
-   ad per variant) is auction-driven: do not run SRM as a
-   gate. Report delivery imbalance as a caveat and compare
-   rates, not totals. Run SRM → Invalid only when traffic
-   was randomized with a planned split (platform A/B /
-   split-test, or an explicit planned allocation) and
-   χ² p < 0.001. If the setup check fails, give the verdict
-   Invalid immediately, Stop, and recommend pausing or fixing the
-   test (a pause is a later yes via
-   `omneky-launch-and-connect`; never mutate here).
-5. Sample size: state impressions, clicks, and conversions per
-   arm. Thin if either arm has fewer than about 1,000 impressions
-   or 30 clicks (or 10 conversions when the metric is CVR) →
-   Insufficient data. Do not declare a winner.
-6. Horizon not reached and setup check passed → **INTERIM**
-   status only. No statistical verdict (Winner / Not
-   commercially meaningful / Inconclusive / No detectable
-   difference). Do not relabel a failed setup check as
-   INTERIM. If they insist on early stopping, require an
-   explicit acknowledgement and apply Bonferroni across
-   looks or O'Brien-Fleming-style (see the statistics
-   reference). If they will not acknowledge, say early
-   stopping is not supported.
-7. Horizon reached (or corrected early stop): two-proportion
-   z-test on the precommitted metric. Two-sided. Show n,
-   successes, z, p-value, 95% CI, achieved power. This is a
-   simple frequentist check, not an uncorrected sequential test
-   — say that.
-8. Verdict:
-   - **Invalid** — exempt from the horizon gate. From the
-     step-4 setup check on every look, including peeks.
-     Immediate Stop. Recommend pausing or fixing (later yes
-     via `omneky-launch-and-connect`; never mutate here).
-   - Only these wait for `horizon` (or a corrected early
-     stop — lead with this):
-     - **Inconclusive (underpowered)** — power < 0.8 and not
-       significant
-     - **No detectable difference** — |z| < 1.96 and power ≥ 0.8
-     - Not commercially meaningful — significant but |lift| < `mde`
-       (or $ impact below the stated floor)
-     - **Winner** — significant, |lift| ≥ `mde`, $ impact stated
-9. Scale / kill / iterate is a later yes on
-   `omneky-launch-and-connect`. Do not mutate here.
+**Create**
+1. Write one hypothesis with one changed variable.
+2. Record metric, MDE, duration (`horizon`) and stop rule before launch.
+3. Budget at or above the floor. One ad group with one ad per variant
+   does not guarantee an even spend split (delivery is auction-driven);
+   say so. For an even split, use one ad group (or campaign) per variant
+   with equal budgets, or the platform's A/B / split-test feature.
+4. Launch through `omneky-launch-and-connect`, live or paused as the
+   user chose (the host confirms).
+
+**Measure**
+1. Confirm coverage for the window; empty is missing data, not zero.
+2. Run the setup check on every look (broken setup; SRM only for a
+   randomized planned allocation, not default auction delivery). If it
+   fails, give the verdict Invalid at once and recommend a pause.
+   Never mutate here.
+3. Before `horizon`, report INTERIM with no statistical verdict.
+4. At `horizon`, run the z-test and give one verdict: Winner, Not
+   commercially meaningful, Inconclusive, or No detectable difference.
+5. Refuse a winner when the sample is thin.
 
 ## Failure and recovery
 
 Auth / 401 → `omneky-getting-started-and-failures` (host OAuth).
-Empty reporting → empty is missing data, not zero; narrow the window.
-`credit_insufficient` → nothing charged; the user can add credits in the Omneky app.
-`needs_user_decision` → `request_user_decision`; never dump JSON.
-Missing tool → say so; Omneky app. Do not invent it.
+Empty reporting → `get_performance_summary` (`include_coverage=true`); empty is not zero.
+`credit_insufficient` → nothing charged; state balance vs required
+(`omneky-billing`). No payment links.
+`needs_user_decision` → fill the missing detail and call again; ask in plain chat only if you cannot. Never dump JSON.
+Missing tool → this session may run an older published app snapshot: look
+the name up in [tool-name-fallbacks](../_shared/references/tool-name-fallbacks.md)
+and call the older name if that one is on `tools/list`. If neither is
+listed, say so; Omneky app. Do not invent it.
 Never ask for internal ids. Never silent-switch `brand_id`.
 
 ## Gaps
 
-This connector cannot do comment/social inbox, landing pages,
-subtitle burn-in, email/SMS, organic scheduling, Meta Ad Library
-competitor ads, virtual try-on, or website builder. Decline those.
-See [capability-gaps](../_shared/references/capability-gaps.md).
+Decline social inbox, landing pages, subtitle burn-in, email/SMS,
+organic scheduling, Ad Library ads, try-on, and website builder
+([capability-gaps](../_shared/references/capability-gaps.md)).
 There is no sequential-testing platform tool. Early stop is
 acknowledgement + correction, or not supported.
Full technical diff · 3 changed fields

changed /description

BEFORE
"Create or measure an Omneky paid-media A/B test. Use for A/B test creation (one hypothesis, one variable, precommitted plan, PAUSED launch) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), first-run onboarding (omneky-onboarding), agency (omneky-agency), weekly marketing plans (omneky-marketing-plan), generating creatives (omneky-image-ads, omneky-product-video), or billing reads (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proportion z-test, MDE, SRM, interim peek, underpowered, commercially meaningful, budget split."
AFTER
"Create or measure an Omneky paid-media A/B test. Use for A/B test creation (one hypothesis, one variable, precommitted plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), first-run onboarding (omneky-onboarding), agency (omneky-agency), weekly marketing plans (omneky-marketing-plan), generating creatives (omneky-image-ads, omneky-product-video, omneky-avatar-ugc), or credit balance (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proportion z-test, MDE, SRM, interim peek, underpowered, commercially meaningful, budget split."

changed /included_files

BEFORE
[]
AFTER
[
  {
    "relative_path": "references/playbook.md",
    "size_in_bytes": 7149
  }
]

changed /skill_md_contents

BEFORE
"---\nname: omneky-ab-tests\ndescription: >-\n  Create or measure an Omneky paid-media A/B test. Use for A/B\n  test creation (one hypothesis, one variable, precommitted\n  plan, PAUSED launch) or A/B test measurement (z-test, CI,\n  power, SRM, interim peek with no verdict, refuse a thin\n  winner). Do not use for generic ROAS tables\n  (omneky-analytics-and-catalogue), competitor analysis\n  (omneky-competitor-analysis), first-run onboarding\n  (omneky-onboarding), agency (omneky-agency), weekly marketing\n  plans (omneky-marketing-plan), generating creatives\n  (omneky-image-ads, omneky-product-video), or billing reads\n  (omneky-billing). Keywords: A/B test, ab test create, ab test\n  measurement, which variant won, significance, two-proportion\n  z-test, MDE, SRM, interim peek, underpowered, commercially\n  meaningful, budget split.\n---\n\n# A/B tests\n\nUse the connected Omneky MCP server. Gate every named tool on this\nsession's `tools/list`. If the tool for a requested step is missing,\nsay so and point to the Omneky app. Do not invent a tool.\n\nThis skill owns the **test plan** (create) and the **readout**\n(measure). Generic winner lists stay in\n`omneky-analytics-and-catalogue`. A one-off `PAUSED` launch without\na test plan stays in `omneky-launch-and-connect`.\n\nHandoff fields:\n[handoff-contract](../_shared/references/handoff-contract.md)\n(including the A/B test plan fields).\nFormulas and worked examples (example data only):\n[ab-test-statistics](../_shared/references/ab-test-statistics.md).\nSourced ids:\n[preconditions](../_shared/references/preconditions.md).\nAcceptance gates:\n[creative-quality-gates](../_shared/references/creative-quality-gates.md).\nStructured brief:\n[creative-brief](../_shared/references/creative-brief.md).\nOther intents:\n[capability-registry](../_shared/references/capability-registry.md).\nWrite for:\n[marketer-persona](../_shared/references/marketer-persona.md).\n\n## Goal\n\nCreate a one-variable `PAUSED` A/B test with a precommitted plan,\nor measure it with a two-proportion z-test and a marketer verdict.\n\n## When to use\n\n- A/B test create, A/B test measurement, which variant won,\n  significance, stop rule, interim peek, SRM, MDE\n\n## When not to use\n\n- Generic ROAS table → `omneky-analytics-and-catalogue`\n- Launch without a test plan → `omneky-launch-and-connect`\n\n## Prerequisites\n\n| Need | Check | If missing |\n|---|---|---|\n| Brand | `list_brands` | Ask which |\n| Channel | `get_channel_connection_status` | Connect first (`omneky-launch-and-connect`) |\n| Budget floor | `minimum_budget_for_objective` | Do not launch under the floor |\n| Variants | prior creative handoff | Owning creative skill |\n| Baseline | `get_daily_metrics` with `date_range` | State an assumed `baseline_rate` |\n| Coverage (measure) | Metric tools for the window; empty is missing data | Do not declare a winner |\n| Metrics (measure) | `get_performance_breakdown` / `get_daily_metrics` | Narrow the window |\n\n## Flow\n\nFollow Create or Measure. **Invalid** is exempt from the\nhorizon gate. The setup check (step 4) runs on every look,\nincluding pre-horizon peeks. If it fails, give the verdict\nInvalid immediately, Stop, and recommend pausing or fixing the\ntest (a pause is a later yes via `omneky-launch-and-connect`;\nnever mutate here). Only Winner, Not commercially\nmeaningful, Inconclusive, and No detectable difference wait\nfor `horizon`. A peek that passes the setup check is INTERIM\nwith no statistical verdict. Refuse a winner when the sample\nis thin.\n\n## Hard rules\n\n- Stick to the chosen `brand_id`. Never silent-switch brands.\n- One hypothesis, **one** variable. Extra changes are a new test.\n- Default new launches to `PAUSED`. Live spend needs an explicit\n  yes in the same summary.\n- Call the budget-floor tool (`minimum_budget_for_objective`)\n  before any budget. Do not set a daily budget under that floor.\n  Split the approved total across variants; each arm stays at or\n  above the floor or you do not launch.\n- Paid creatives that precede the test use the owning creative\n  skill: Never invent a price.\n- Precommit and record the test plan: `primary_metric` (one),\n  `baseline_rate`, `mde`, `alpha` (default 0.05), `power`\n  (default 0.8), `n_per_arm` from the two-proportion formula\n  (show it), `randomization_unit` plus the ad-level-split caveat,\n  `attribution_window`, and `horizon` (end date or sample).\n- Measure: **Invalid** is exempt from the horizon gate. The\n  setup check runs on every look, including pre-horizon\n  peeks. If it fails, give the verdict Invalid immediately,\n  Stop, and recommend pausing or fixing (pause is a later yes via\n  `omneky-launch-and-connect`; never mutate here). Only\n  Winner, Not commercially meaningful, Inconclusive, and\n  No detectable difference wait for `horizon`. A peek that\n  passes setup is INTERIM. Early stop needs an explicit\n  acknowledgement and a Bonferroni or O'Brien-Fleming\n  correction (or say it is not supported).\n- Report effect size + CI, p-value, and achieved power.\n  Significance is not commercial meaning — compare to `mde` and\n  a CPA / ROAS $ impact.\n- SRM → Invalid only on a randomized planned split (platform\n  A/B / split-test, or an explicit planned allocation). Default\n  one-ad-group, one-ad-per-variant delivery is auction-driven:\n  do not run SRM as a gate; report imbalance as a caveat and\n  compare rates, not totals. Broken setup (near-zero delivery,\n  mid-test change, more than one variable) → Invalid. Low\n  counts → Insufficient data. Many variants → Bonferroni.\n- Scale / kill / iterate needs a later yes\n  (`omneky-launch-and-connect`).\n\n## Create\n\n1. Write the test plan with the user (one turn if they already\n   gave it):\n   - Hypothesis (what should win, and why)\n   - Single variable (creative, or headline, or audience — pick one)\n   - `primary_metric` (CTR or CVR — one only)\n   - `baseline_rate` from `get_daily_metrics` with `date_range`,\n     or a stated assumption\n   - `mde` (absolute rate)\n   - `alpha` 0.05 and `power` 0.8 unless they set others\n   - `n_per_arm` from the two-proportion formula (show it;\n     [ab-test-statistics](../_shared/references/ab-test-statistics.md))\n   - `randomization_unit` and the caveat that platform ad-level\n     splits are not true user randomization unless platform A/B\n     is used\n   - `attribution_window`\n   - `horizon` (end date or sample). This is the stop rule.\n2. Variants come from the structured brief (or a prior creative\n   handoff). Same brief except the one variable.\n3. Structure: one campaign, one ad group, one ad per variant\n   (same targeting and budget split). Use an existing campaign\n   only when they named it (`campaign_spec.id`).\n4. Preflight the channel with `omneky-launch-and-connect` rules:\n   connect status, budget floor, targeting pre-call, copy / CTA /\n   landing URL, credits. Default new launches to `PAUSED`.\n5. One summary: channel, tool, variants, budget split, floor,\n   the full test plan, `PAUSED`. Wait for one yes.\n6. Call the matching `launch_*`. Then write the test plan back\n   (`primary_metric`, `baseline_rate`, `mde`, `alpha`, `power`,\n   `n_per_arm`, `randomization_unit`, `attribution_window`,\n   `horizon`, campaign / ad ids).\n\nDo not go live unless that yes said to go live.\n\n## Measure\n\n1. Reuse the test plan. Missing ids: `get_campaigns` /\n   `get_ad_groups` (pass `campaign_id`) / `get_ads`. Do not guess.\n2. Empty metrics for the brand + window + channel mean no imported\n   data → stop. Empty is not zero.\n3. `get_performance_breakdown` (dimension ad or creative) and\n   `get_daily_metrics` for the window. `get_performance_movers`\n   when they asked what changed vs the prior period.\n4. Setup check on every look, including pre-horizon peeks.\n   **Invalid** only for a broken setup: near-zero delivery\n   on a variant, a variant changed mid-test, or more than\n   one variable differs. Default setup (one ad group, one\n   ad per variant) is auction-driven: do not run SRM as a\n   gate. Report delivery imbalance as a caveat and compare\n   rates, not totals. Run SRM → Invalid only when traffic\n   was randomized with a planned split (platform A/B /\n   split-test, or an explicit planned allocation) and\n   χ² p < 0.001. If the setup check fails, give the verdict\n   Invalid immediately, Stop, and recommend pausing or fixing the\n   test (a pause is a later yes via\n   `omneky-launch-and-connect`; never mutate here).\n5. Sample size: state impressions, clicks, and conversions per\n   arm. Thin if either arm has fewer than about 1,000 impressions\n   or 30 clicks (or 10 conversions when the metric is CVR) →\n   Insufficient data. Do not declare a winner.\n6. Horizon not reached and setup check passed → **INTERIM**\n   status only. No statistical verdict (Winner / Not\n   commercially meaningful / Inconclusive / No detectable\n   difference). Do not relabel a failed setup check as\n   INTERIM. If they insist on early stopping, require an\n   explicit acknowledgement and apply Bonferroni across\n   looks or O'Brien-Fleming-style (see the statistics\n   reference). If they will not acknowledge, say early\n   stopping is not supported.\n7. Horizon reached (or corrected early stop): two-proportion\n   z-test on the precommitted metric. Two-sided. Show n,\n   successes, z, p-value, 95% CI, achieved power. This is a\n   simple frequentist check, not an uncorrected sequential test\n   — say that.\n8. Verdict:\n   - **Invalid** — exempt from the horizon gate. From the\n     step-4 setup check on every look, including peeks.\n     Immediate Stop. Recommend pausing or fixing (later yes\n     via `omneky-launch-and-connect`; never mutate here).\n   - Only these wait for `horizon` (or a corrected early\n     stop — lead with this):\n     - **Inconclusive (underpowered)** — power < 0.8 and not\n       significant\n     - **No detectable difference** — |z| < 1.96 and power ≥ 0.8\n     - Not commercially meaningful — significant but |lift| < `mde`\n       (or $ impact below the stated floor)\n     - **Winner** — significant, |lift| ≥ `mde`, $ impact stated\n9. Scale / kill / iterate is a later yes on\n   `omneky-launch-and-connect`. Do not mutate here.\n\n## Failure and recovery\n\nAuth / 401 → `omneky-getting-started-and-failures` (host OAuth).\nEmpty reporting → empty is missing data, not zero; narrow the window.\n`credit_insufficient` → nothing charged; the user can add credits in the Omneky app.\n`needs_user_decision` → `request_user_decision`; never dump JSON.\nMissing tool → say so; Omneky app. Do not invent it.\nNever ask for internal ids. Never silent-switch `brand_id`.\n\n## Gaps\n\nThis connector cannot do comment/social inbox, landing pages,\nsubtitle burn-in, email/SMS, organic scheduling, Meta Ad Library\ncompetitor ads, virtual try-on, or website builder. Decline those.\nSee [capability-gaps](../_shared/references/capability-gaps.md).\nThere is no sequential-testing platform tool. Early stop is\nacknowledgement + correction, or not supported.\n"
AFTER
"---\nname: omneky-ab-tests\ndescription: >-\n  Create or measure an Omneky paid-media A/B test. Use for A/B\n  test creation (one hypothesis, one variable, precommitted\n  plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI,\n  power, SRM, interim peek with no verdict, refuse a thin\n  winner). Do not use for generic ROAS tables\n  (omneky-analytics-and-catalogue), competitor analysis\n  (omneky-competitor-analysis), first-run onboarding\n  (omneky-onboarding), agency (omneky-agency), weekly marketing\n  plans (omneky-marketing-plan), generating creatives\n  (omneky-image-ads, omneky-product-video, omneky-avatar-ugc),\n  or credit balance (omneky-billing). Keywords: A/B test,\n  ab test create, ab test measurement, which variant won,\n  significance, two-proportion z-test, MDE, SRM, interim peek,\n  underpowered, commercially meaningful, budget split.\n---\n\n# A/B tests\n\nUse this when the user wants to set up an A/B test or read out which\nvariant won. One job: own the test plan and the readout.\n\nGate every tool on this session's `tools/list`. If a step's tool is\nmissing, say so and point to the Omneky app.\n\nPlan fields: [handoff-contract](../_shared/references/handoff-contract.md).\nFormulas: [ab-test-statistics](../_shared/references/ab-test-statistics.md).\nSourced ids: [preconditions](../_shared/references/preconditions.md).\nAcceptance gates: [creative-quality-gates](../_shared/references/creative-quality-gates.md).\nBrief: [creative-brief](../_shared/references/creative-brief.md).\nOther intents: [capability-registry](../_shared/references/capability-registry.md).\nWrite for: [marketer-persona](../_shared/references/marketer-persona.md).\nStep detail: [playbook](references/playbook.md) (Hard rules, Create, Measure).\n\n## Goal\n\nCreate a one-variable test with a precommitted plan (live or paused is\nthe user's choice in the host), or\nmeasure it with a two-proportion z-test and a marketer verdict.\n\n## When to use\n\n- Create an A/B test, measure one, \"which variant won\", significance,\n  stop rule, interim peek, SRM, MDE\n\n## When not to use\n\n- Generic ROAS table → `omneky-analytics-and-catalogue`\n- Launch without a test plan → `omneky-launch-and-connect`\n\n## Prerequisites\n\n| Need | Check | If missing |\n|---|---|---|\n| Brand | `list_brands` | Ask which |\n| Channel | `list_connections` (`connector`) | Connect or managed |\n| Budget floor | `get_launch_requirements` | Do not launch under the floor |\n| Variants | prior creative handoff | Owning creative skill |\n| Baseline | `get_performance_summary` with `date_range` | State an assumed `baseline_rate` |\n| Coverage (measure) | `get_performance_summary` (`include_coverage=true`) | Do not declare a winner |\n| Metrics (measure) | `get_performance_breakdown` / `get_performance_summary` | Narrow the window |\n\n## Flow\n\n**Create**\n1. Write one hypothesis with one changed variable.\n2. Record metric, MDE, duration (`horizon`) and stop rule before launch.\n3. Budget at or above the floor. One ad group with one ad per variant\n   does not guarantee an even spend split (delivery is auction-driven);\n   say so. For an even split, use one ad group (or campaign) per variant\n   with equal budgets, or the platform's A/B / split-test feature.\n4. Launch through `omneky-launch-and-connect`, live or paused as the\n   user chose (the host confirms).\n\n**Measure**\n1. Confirm coverage for the window; empty is missing data, not zero.\n2. Run the setup check on every look (broken setup; SRM only for a\n   randomized planned allocation, not default auction delivery). If it\n   fails, give the verdict Invalid at once and recommend a pause.\n   Never mutate here.\n3. Before `horizon`, report INTERIM with no statistical verdict.\n4. At `horizon`, run the z-test and give one verdict: Winner, Not\n   commercially meaningful, Inconclusive, or No detectable difference.\n5. Refuse a winner when the sample is thin.\n\n## Failure and recovery\n\nAuth / 401 → `omneky-getting-started-and-failures` (host OAuth).\nEmpty reporting → `get_performance_summary` (`include_coverage=true`); empty is not zero.\n`credit_insufficient` → nothing charged; state balance vs required\n(`omneky-billing`). No payment links.\n`needs_user_decision` → fill the missing detail and call again; ask in plain chat only if you cannot. Never dump JSON.\nMissing tool → this session may run an older published app snapshot: look\nthe name up in [tool-name-fallbacks](../_shared/references/tool-name-fallbacks.md)\nand call the older name if that one is on `tools/list`. If neither is\nlisted, say so; Omneky app. Do not invent it.\nNever ask for internal ids. Never silent-switch `brand_id`.\n\n## Gaps\n\nDecline social inbox, landing pages, subtitle burn-in, email/SMS,\norganic scheduling, Ad Library ads, try-on, and website builder\n([capability-gaps](../_shared/references/capability-gaps.md)).\nThere is no sequential-testing platform tool. Early stop is\nacknowledgement + correction, or not supported.\n"
Full snapshot data
{
  "description": "Create or measure an Omneky paid-media A/B test. Use for A/B test creation (one hypothesis, one variable, precommitted plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), first-run onboarding (omneky-onboarding), agency (omneky-agency), weekly marketing plans (omneky-marketing-plan), generating creatives (omneky-image-ads, omneky-product-video, omneky-avatar-ugc), or credit balance (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proportion z-test, MDE, SRM, interim peek, underpowered, commercially meaningful, budget split.",
  "included_files": [
    {
      "relative_path": "references/playbook.md",
      "size_in_bytes": 7149
    }
  ],
  "name": "omneky-ab-tests",
  "skill_md_contents": "---\nname: omneky-ab-tests\ndescription: >-\n  Create or measure an Omneky paid-media A/B test. Use for A/B\n  test creation (one hypothesis, one variable, precommitted\n  plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI,\n  power, SRM, interim peek with no verdict, refuse a thin\n  winner). Do not use for generic ROAS tables\n  (omneky-analytics-and-catalogue), competitor analysis\n  (omneky-competitor-analysis), first-run onboarding\n  (omneky-onboarding), agency (omneky-agency), weekly marketing\n  plans (omneky-marketing-plan), generating creatives\n  (omneky-image-ads, omneky-product-video, omneky-avatar-ugc),\n  or credit balance (omneky-billing). Keywords: A/B test,\n  ab test create, ab test measurement, which variant won,\n  significance, two-proportion z-test, MDE, SRM, interim peek,\n  underpowered, commercially meaningful, budget split.\n---\n\n# A/B tests\n\nUse this when the user wants to set up an A/B test or read out which\nvariant won. One job: own the test plan and the readout.\n\nGate every tool on this session's `tools/list`. If a step's tool is\nmissing, say so and point to the Omneky app.\n\nPlan fields: [handoff-contract](../_shared/references/handoff-contract.md).\nFormulas: [ab-test-statistics](../_shared/references/ab-test-statistics.md).\nSourced ids: [preconditions](../_shared/references/preconditions.md).\nAcceptance gates: [creative-quality-gates](../_shared/references/creative-quality-gates.md).\nBrief: [creative-brief](../_shared/references/creative-brief.md).\nOther intents: [capability-registry](../_shared/references/capability-registry.md).\nWrite for: [marketer-persona](../_shared/references/marketer-persona.md).\nStep detail: [playbook](references/playbook.md) (Hard rules, Create, Measure).\n\n## Goal\n\nCreate a one-variable test with a precommitted plan (live or paused is\nthe user's choice in the host), or\nmeasure it with a two-proportion z-test and a marketer verdict.\n\n## When to use\n\n- Create an A/B test, measure one, \"which variant won\", significance,\n  stop rule, interim peek, SRM, MDE\n\n## When not to use\n\n- Generic ROAS table → `omneky-analytics-and-catalogue`\n- Launch without a test plan → `omneky-launch-and-connect`\n\n## Prerequisites\n\n| Need | Check | If missing |\n|---|---|---|\n| Brand | `list_brands` | Ask which |\n| Channel | `list_connections` (`connector`) | Connect or managed |\n| Budget floor | `get_launch_requirements` | Do not launch under the floor |\n| Variants | prior creative handoff | Owning creative skill |\n| Baseline | `get_performance_summary` with `date_range` | State an assumed `baseline_rate` |\n| Coverage (measure) | `get_performance_summary` (`include_coverage=true`) | Do not declare a winner |\n| Metrics (measure) | `get_performance_breakdown` / `get_performance_summary` | Narrow the window |\n\n## Flow\n\n**Create**\n1. Write one hypothesis with one changed variable.\n2. Record metric, MDE, duration (`horizon`) and stop rule before launch.\n3. Budget at or above the floor. One ad group with one ad per variant\n   does not guarantee an even spend split (delivery is auction-driven);\n   say so. For an even split, use one ad group (or campaign) per variant\n   with equal budgets, or the platform's A/B / split-test feature.\n4. Launch through `omneky-launch-and-connect`, live or paused as the\n   user chose (the host confirms).\n\n**Measure**\n1. Confirm coverage for the window; empty is missing data, not zero.\n2. Run the setup check on every look (broken setup; SRM only for a\n   randomized planned allocation, not default auction delivery). If it\n   fails, give the verdict Invalid at once and recommend a pause.\n   Never mutate here.\n3. Before `horizon`, report INTERIM with no statistical verdict.\n4. At `horizon`, run the z-test and give one verdict: Winner, Not\n   commercially meaningful, Inconclusive, or No detectable difference.\n5. Refuse a winner when the sample is thin.\n\n## Failure and recovery\n\nAuth / 401 → `omneky-getting-started-and-failures` (host OAuth).\nEmpty reporting → `get_performance_summary` (`include_coverage=true`); empty is not zero.\n`credit_insufficient` → nothing charged; state balance vs required\n(`omneky-billing`). No payment links.\n`needs_user_decision` → fill the missing detail and call again; ask in plain chat only if you cannot. Never dump JSON.\nMissing tool → this session may run an older published app snapshot: look\nthe name up in [tool-name-fallbacks](../_shared/references/tool-name-fallbacks.md)\nand call the older name if that one is on `tools/list`. If neither is\nlisted, say so; Omneky app. Do not invent it.\nNever ask for internal ids. Never silent-switch `brand_id`.\n\n## Gaps\n\nDecline social inbox, landing pages, subtitle burn-in, email/SMS,\norganic scheduling, Ad Library ads, try-on, and website builder\n([capability-gaps](../_shared/references/capability-gaps.md)).\nThere is no sequential-testing platform tool. Early stop is\nacknowledgement + correction, or not supported.\n"
}

SHA-256 of public snapshot: 98fbd2470c67614f02944540754b9bc2813e76da7627c4ad065cd2a3a21bae89