Update to Omneky
Snapshot Oct 27, 2026 · 00:28 UTC · version 2.0.31
Collection source: downloaded plugin package.
Payment or plan references changed
Full description — Now says: launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use...
Observed in published text. Live prices and checkout terms have not been verified by this change.
Up to 3 plugins free · no card required.
All fields and original values.
Full description
Oct 24, 2026 · 13:12 UTC
PAUSED launch) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), f...
Oct 27, 2026 · 00:28 UTC
launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (om...
Skill instructions
Oct 24, 2026 · 13:12 UTC
PAUSED launch) or A/B test measurement (z-test, CI, (omneky-image-ads, omneky-product-video), or billing reads (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proporti...
Oct 27, 2026 · 00:28 UTC
launch live or paused as the user chooses) or A/B test measurement (z-test, CI, (omneky-image-ads, omneky-product-video, omneky-avatar-ugc), or credit balance (omneky-billing). Keywords: A/B test, ab test create, ab test measuremen...
Supporting files
Oct 24, 2026 · 13:12 UTC
[]
Oct 27, 2026 · 00:28 UTC
[{"relative_path":"references/playbook.md","size_in_bytes":7149}]
Compare saved observations
Download comparison JSONSKILL.md text changes
− removed · + added
--- before +++ after @@ -3,56 +3,47 @@ description: >- Create or measure an Omneky paid-media A/B test. Use for A/B test creation (one hypothesis, one variable, precommitted - plan, PAUSED launch) or A/B test measurement (z-test, CI, + plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), first-run onboarding (omneky-onboarding), agency (omneky-agency), weekly marketing plans (omneky-marketing-plan), generating creatives - (omneky-image-ads, omneky-product-video), or billing reads - (omneky-billing). Keywords: A/B test, ab test create, ab test - measurement, which variant won, significance, two-proportion - z-test, MDE, SRM, interim peek, underpowered, commercially - meaningful, budget split. + (omneky-image-ads, omneky-product-video, omneky-avatar-ugc), + or credit balance (omneky-billing). Keywords: A/B test, + ab test create, ab test measurement, which variant won, + significance, two-proportion z-test, MDE, SRM, interim peek, + underpowered, commercially meaningful, budget split. --- # A/B tests -Use the connected Omneky MCP server. Gate every named tool on this -session's `tools/list`. If the tool for a requested step is missing, -say so and point to the Omneky app. Do not invent a tool. - -This skill owns the **test plan** (create) and the **readout** -(measure). Generic winner lists stay in -`omneky-analytics-and-catalogue`. A one-off `PAUSED` launch without -a test plan stays in `omneky-launch-and-connect`. - -Handoff fields: -[handoff-contract](../_shared/references/handoff-contract.md) -(including the A/B test plan fields). -Formulas and worked examples (example data only): -[ab-test-statistics](../_shared/references/ab-test-statistics.md). -Sourced ids: -[preconditions](../_shared/references/preconditions.md). -Acceptance gates: -[creative-quality-gates](../_shared/references/creative-quality-gates.md). -Structured brief: -[creative-brief](../_shared/references/creative-brief.md). -Other intents: -[capability-registry](../_shared/references/capability-registry.md). -Write for: -[marketer-persona](../_shared/references/marketer-persona.md). +Use this when the user wants to set up an A/B test or read out which +variant won. One job: own the test plan and the readout. + +Gate every tool on this session's `tools/list`. If a step's tool is +missing, say so and point to the Omneky app. + +Plan fields: [handoff-contract](../_shared/references/handoff-contract.md). +Formulas: [ab-test-statistics](../_shared/references/ab-test-statistics.md). +Sourced ids: [preconditions](../_shared/references/preconditions.md). +Acceptance gates: [creative-quality-gates](../_shared/references/creative-quality-gates.md). +Brief: [creative-brief](../_shared/references/creative-brief.md). +Other intents: [capability-registry](../_shared/references/capability-registry.md). +Write for: [marketer-persona](../_shared/references/marketer-persona.md). +Step detail: [playbook](references/playbook.md) (Hard rules, Create, Measure). ## Goal -Create a one-variable `PAUSED` A/B test with a precommitted plan, -or measure it with a two-proportion z-test and a marketer verdict. +Create a one-variable test with a precommitted plan (live or paused is +the user's choice in the host), or +measure it with a two-proportion z-test and a marketer verdict. ## When to use -- A/B test create, A/B test measurement, which variant won, - significance, stop rule, interim peek, SRM, MDE +- Create an A/B test, measure one, "which variant won", significance, + stop rule, interim peek, SRM, MDE ## When not to use @@ -64,171 +55,53 @@ | Need | Check | If missing | |---|---|---| | Brand | `list_brands` | Ask which | -| Channel | `get_channel_connection_status` | Connect first (`omneky-launch-and-connect`) | -| Budget floor | `minimum_budget_for_objective` | Do not launch under the floor | +| Channel | `list_connections` (`connector`) | Connect or managed | +| Budget floor | `get_launch_requirements` | Do not launch under the floor | | Variants | prior creative handoff | Owning creative skill | -| Baseline | `get_daily_metrics` with `date_range` | State an assumed `baseline_rate` | -| Coverage (measure) | Metric tools for the window; empty is missing data | Do not declare a winner | -| Metrics (measure) | `get_performance_breakdown` / `get_daily_metrics` | Narrow the window | +| Baseline | `get_performance_summary` with `date_range` | State an assumed `baseline_rate` | +| Coverage (measure) | `get_performance_summary` (`include_coverage=true`) | Do not declare a winner | +| Metrics (measure) | `get_performance_breakdown` / `get_performance_summary` | Narrow the window | ## Flow -Follow Create or Measure. **Invalid** is exempt from the -horizon gate. The setup check (step 4) runs on every look, -including pre-horizon peeks. If it fails, give the verdict -Invalid immediately, Stop, and recommend pausing or fixing the -test (a pause is a later yes via `omneky-launch-and-connect`; -never mutate here). Only Winner, Not commercially -meaningful, Inconclusive, and No detectable difference wait -for `horizon`. A peek that passes the setup check is INTERIM -with no statistical verdict. Refuse a winner when the sample -is thin. - -## Hard rules - -- Stick to the chosen `brand_id`. Never silent-switch brands. -- One hypothesis, **one** variable. Extra changes are a new test. -- Default new launches to `PAUSED`. Live spend needs an explicit - yes in the same summary. -- Call the budget-floor tool (`minimum_budget_for_objective`) - before any budget. Do not set a daily budget under that floor. - Split the approved total across variants; each arm stays at or - above the floor or you do not launch. -- Paid creatives that precede the test use the owning creative - skill: Never invent a price. -- Precommit and record the test plan: `primary_metric` (one), - `baseline_rate`, `mde`, `alpha` (default 0.05), `power` - (default 0.8), `n_per_arm` from the two-proportion formula - (show it), `randomization_unit` plus the ad-level-split caveat, - `attribution_window`, and `horizon` (end date or sample). -- Measure: **Invalid** is exempt from the horizon gate. The - setup check runs on every look, including pre-horizon - peeks. If it fails, give the verdict Invalid immediately, - Stop, and recommend pausing or fixing (pause is a later yes via - `omneky-launch-and-connect`; never mutate here). Only - Winner, Not commercially meaningful, Inconclusive, and - No detectable difference wait for `horizon`. A peek that - passes setup is INTERIM. Early stop needs an explicit - acknowledgement and a Bonferroni or O'Brien-Fleming - correction (or say it is not supported). -- Report effect size + CI, p-value, and achieved power. - Significance is not commercial meaning — compare to `mde` and - a CPA / ROAS $ impact. -- SRM → Invalid only on a randomized planned split (platform - A/B / split-test, or an explicit planned allocation). Default - one-ad-group, one-ad-per-variant delivery is auction-driven: - do not run SRM as a gate; report imbalance as a caveat and - compare rates, not totals. Broken setup (near-zero delivery, - mid-test change, more than one variable) → Invalid. Low - counts → Insufficient data. Many variants → Bonferroni. -- Scale / kill / iterate needs a later yes - (`omneky-launch-and-connect`). - -## Create - -1. Write the test plan with the user (one turn if they already - gave it): - - Hypothesis (what should win, and why) - - Single variable (creative, or headline, or audience — pick one) - - `primary_metric` (CTR or CVR — one only) - - `baseline_rate` from `get_daily_metrics` with `date_range`, - or a stated assumption - - `mde` (absolute rate) - - `alpha` 0.05 and `power` 0.8 unless they set others - - `n_per_arm` from the two-proportion formula (show it; - [ab-test-statistics](../_shared/references/ab-test-statistics.md)) - - `randomization_unit` and the caveat that platform ad-level - splits are not true user randomization unless platform A/B - is used - - `attribution_window` - - `horizon` (end date or sample). This is the stop rule. -2. Variants come from the structured brief (or a prior creative - handoff). Same brief except the one variable. -3. Structure: one campaign, one ad group, one ad per variant - (same targeting and budget split). Use an existing campaign - only when they named it (`campaign_spec.id`). -4. Preflight the channel with `omneky-launch-and-connect` rules: - connect status, budget floor, targeting pre-call, copy / CTA / - landing URL, credits. Default new launches to `PAUSED`. -5. One summary: channel, tool, variants, budget split, floor, - the full test plan, `PAUSED`. Wait for one yes. -6. Call the matching `launch_*`. Then write the test plan back - (`primary_metric`, `baseline_rate`, `mde`, `alpha`, `power`, - `n_per_arm`, `randomization_unit`, `attribution_window`, - `horizon`, campaign / ad ids). - -Do not go live unless that yes said to go live. - -## Measure - -1. Reuse the test plan. Missing ids: `get_campaigns` / - `get_ad_groups` (pass `campaign_id`) / `get_ads`. Do not guess. -2. Empty metrics for the brand + window + channel mean no imported - data → stop. Empty is not zero. -3. `get_performance_breakdown` (dimension ad or creative) and - `get_daily_metrics` for the window. `get_performance_movers` - when they asked what changed vs the prior period. -4. Setup check on every look, including pre-horizon peeks. - **Invalid** only for a broken setup: near-zero delivery - on a variant, a variant changed mid-test, or more than - one variable differs. Default setup (one ad group, one - ad per variant) is auction-driven: do not run SRM as a - gate. Report delivery imbalance as a caveat and compare - rates, not totals. Run SRM → Invalid only when traffic - was randomized with a planned split (platform A/B / - split-test, or an explicit planned allocation) and - χ² p < 0.001. If the setup check fails, give the verdict - Invalid immediately, Stop, and recommend pausing or fixing the - test (a pause is a later yes via - `omneky-launch-and-connect`; never mutate here). -5. Sample size: state impressions, clicks, and conversions per - arm. Thin if either arm has fewer than about 1,000 impressions - or 30 clicks (or 10 conversions when the metric is CVR) → - Insufficient data. Do not declare a winner. -6. Horizon not reached and setup check passed → **INTERIM** - status only. No statistical verdict (Winner / Not - commercially meaningful / Inconclusive / No detectable - difference). Do not relabel a failed setup check as - INTERIM. If they insist on early stopping, require an - explicit acknowledgement and apply Bonferroni across - looks or O'Brien-Fleming-style (see the statistics - reference). If they will not acknowledge, say early - stopping is not supported. -7. Horizon reached (or corrected early stop): two-proportion - z-test on the precommitted metric. Two-sided. Show n, - successes, z, p-value, 95% CI, achieved power. This is a - simple frequentist check, not an uncorrected sequential test - — say that. -8. Verdict: - - **Invalid** — exempt from the horizon gate. From the - step-4 setup check on every look, including peeks. - Immediate Stop. Recommend pausing or fixing (later yes - via `omneky-launch-and-connect`; never mutate here). - - Only these wait for `horizon` (or a corrected early - stop — lead with this): - - **Inconclusive (underpowered)** — power < 0.8 and not - significant - - **No detectable difference** — |z| < 1.96 and power ≥ 0.8 - - Not commercially meaningful — significant but |lift| < `mde` - (or $ impact below the stated floor) - - **Winner** — significant, |lift| ≥ `mde`, $ impact stated -9. Scale / kill / iterate is a later yes on - `omneky-launch-and-connect`. Do not mutate here. +**Create** +1. Write one hypothesis with one changed variable. +2. Record metric, MDE, duration (`horizon`) and stop rule before launch. +3. Budget at or above the floor. One ad group with one ad per variant + does not guarantee an even spend split (delivery is auction-driven); + say so. For an even split, use one ad group (or campaign) per variant + with equal budgets, or the platform's A/B / split-test feature. +4. Launch through `omneky-launch-and-connect`, live or paused as the + user chose (the host confirms). + +**Measure** +1. Confirm coverage for the window; empty is missing data, not zero. +2. Run the setup check on every look (broken setup; SRM only for a + randomized planned allocation, not default auction delivery). If it + fails, give the verdict Invalid at once and recommend a pause. + Never mutate here. +3. Before `horizon`, report INTERIM with no statistical verdict. +4. At `horizon`, run the z-test and give one verdict: Winner, Not + commercially meaningful, Inconclusive, or No detectable difference. +5. Refuse a winner when the sample is thin. ## Failure and recovery Auth / 401 → `omneky-getting-started-and-failures` (host OAuth). -Empty reporting → empty is missing data, not zero; narrow the window. -`credit_insufficient` → nothing charged; the user can add credits in the Omneky app. -`needs_user_decision` → `request_user_decision`; never dump JSON. -Missing tool → say so; Omneky app. Do not invent it. +Empty reporting → `get_performance_summary` (`include_coverage=true`); empty is not zero. +`credit_insufficient` → nothing charged; state balance vs required +(`omneky-billing`). No payment links. +`needs_user_decision` → fill the missing detail and call again; ask in plain chat only if you cannot. Never dump JSON. +Missing tool → this session may run an older published app snapshot: look +the name up in [tool-name-fallbacks](../_shared/references/tool-name-fallbacks.md) +and call the older name if that one is on `tools/list`. If neither is +listed, say so; Omneky app. Do not invent it. Never ask for internal ids. Never silent-switch `brand_id`. ## Gaps -This connector cannot do comment/social inbox, landing pages, -subtitle burn-in, email/SMS, organic scheduling, Meta Ad Library -competitor ads, virtual try-on, or website builder. Decline those. -See [capability-gaps](../_shared/references/capability-gaps.md). +Decline social inbox, landing pages, subtitle burn-in, email/SMS, +organic scheduling, Ad Library ads, try-on, and website builder +([capability-gaps](../_shared/references/capability-gaps.md)). There is no sequential-testing platform tool. Early stop is acknowledgement + correction, or not supported.
Full technical diff · 3 changed fields
changed /description
"Create or measure an Omneky paid-media A/B test. Use for A/B test creation (one hypothesis, one variable, precommitted plan, PAUSED launch) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), first-run onboarding (omneky-onboarding), agency (omneky-agency), weekly marketing plans (omneky-marketing-plan), generating creatives (omneky-image-ads, omneky-product-video), or billing reads (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proportion z-test, MDE, SRM, interim peek, underpowered, commercially meaningful, budget split."
"Create or measure an Omneky paid-media A/B test. Use for A/B test creation (one hypothesis, one variable, precommitted plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), first-run onboarding (omneky-onboarding), agency (omneky-agency), weekly marketing plans (omneky-marketing-plan), generating creatives (omneky-image-ads, omneky-product-video, omneky-avatar-ugc), or credit balance (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proportion z-test, MDE, SRM, interim peek, underpowered, commercially meaningful, budget split."
changed /included_files
[]
[
{
"relative_path": "references/playbook.md",
"size_in_bytes": 7149
}
]changed /skill_md_contents
"---\nname: omneky-ab-tests\ndescription: >-\n Create or measure an Omneky paid-media A/B test. Use for A/B\n test creation (one hypothesis, one variable, precommitted\n plan, PAUSED launch) or A/B test measurement (z-test, CI,\n power, SRM, interim peek with no verdict, refuse a thin\n winner). Do not use for generic ROAS tables\n (omneky-analytics-and-catalogue), competitor analysis\n (omneky-competitor-analysis), first-run onboarding\n (omneky-onboarding), agency (omneky-agency), weekly marketing\n plans (omneky-marketing-plan), generating creatives\n (omneky-image-ads, omneky-product-video), or billing reads\n (omneky-billing). Keywords: A/B test, ab test create, ab test\n measurement, which variant won, significance, two-proportion\n z-test, MDE, SRM, interim peek, underpowered, commercially\n meaningful, budget split.\n---\n\n# A/B tests\n\nUse the connected Omneky MCP server. Gate every named tool on this\nsession's `tools/list`. If the tool for a requested step is missing,\nsay so and point to the Omneky app. Do not invent a tool.\n\nThis skill owns the **test plan** (create) and the **readout**\n(measure). Generic winner lists stay in\n`omneky-analytics-and-catalogue`. A one-off `PAUSED` launch without\na test plan stays in `omneky-launch-and-connect`.\n\nHandoff fields:\n[handoff-contract](../_shared/references/handoff-contract.md)\n(including the A/B test plan fields).\nFormulas and worked examples (example data only):\n[ab-test-statistics](../_shared/references/ab-test-statistics.md).\nSourced ids:\n[preconditions](../_shared/references/preconditions.md).\nAcceptance gates:\n[creative-quality-gates](../_shared/references/creative-quality-gates.md).\nStructured brief:\n[creative-brief](../_shared/references/creative-brief.md).\nOther intents:\n[capability-registry](../_shared/references/capability-registry.md).\nWrite for:\n[marketer-persona](../_shared/references/marketer-persona.md).\n\n## Goal\n\nCreate a one-variable `PAUSED` A/B test with a precommitted plan,\nor measure it with a two-proportion z-test and a marketer verdict.\n\n## When to use\n\n- A/B test create, A/B test measurement, which variant won,\n significance, stop rule, interim peek, SRM, MDE\n\n## When not to use\n\n- Generic ROAS table → `omneky-analytics-and-catalogue`\n- Launch without a test plan → `omneky-launch-and-connect`\n\n## Prerequisites\n\n| Need | Check | If missing |\n|---|---|---|\n| Brand | `list_brands` | Ask which |\n| Channel | `get_channel_connection_status` | Connect first (`omneky-launch-and-connect`) |\n| Budget floor | `minimum_budget_for_objective` | Do not launch under the floor |\n| Variants | prior creative handoff | Owning creative skill |\n| Baseline | `get_daily_metrics` with `date_range` | State an assumed `baseline_rate` |\n| Coverage (measure) | Metric tools for the window; empty is missing data | Do not declare a winner |\n| Metrics (measure) | `get_performance_breakdown` / `get_daily_metrics` | Narrow the window |\n\n## Flow\n\nFollow Create or Measure. **Invalid** is exempt from the\nhorizon gate. The setup check (step 4) runs on every look,\nincluding pre-horizon peeks. If it fails, give the verdict\nInvalid immediately, Stop, and recommend pausing or fixing the\ntest (a pause is a later yes via `omneky-launch-and-connect`;\nnever mutate here). Only Winner, Not commercially\nmeaningful, Inconclusive, and No detectable difference wait\nfor `horizon`. A peek that passes the setup check is INTERIM\nwith no statistical verdict. Refuse a winner when the sample\nis thin.\n\n## Hard rules\n\n- Stick to the chosen `brand_id`. Never silent-switch brands.\n- One hypothesis, **one** variable. Extra changes are a new test.\n- Default new launches to `PAUSED`. Live spend needs an explicit\n yes in the same summary.\n- Call the budget-floor tool (`minimum_budget_for_objective`)\n before any budget. Do not set a daily budget under that floor.\n Split the approved total across variants; each arm stays at or\n above the floor or you do not launch.\n- Paid creatives that precede the test use the owning creative\n skill: Never invent a price.\n- Precommit and record the test plan: `primary_metric` (one),\n `baseline_rate`, `mde`, `alpha` (default 0.05), `power`\n (default 0.8), `n_per_arm` from the two-proportion formula\n (show it), `randomization_unit` plus the ad-level-split caveat,\n `attribution_window`, and `horizon` (end date or sample).\n- Measure: **Invalid** is exempt from the horizon gate. The\n setup check runs on every look, including pre-horizon\n peeks. If it fails, give the verdict Invalid immediately,\n Stop, and recommend pausing or fixing (pause is a later yes via\n `omneky-launch-and-connect`; never mutate here). Only\n Winner, Not commercially meaningful, Inconclusive, and\n No detectable difference wait for `horizon`. A peek that\n passes setup is INTERIM. Early stop needs an explicit\n acknowledgement and a Bonferroni or O'Brien-Fleming\n correction (or say it is not supported).\n- Report effect size + CI, p-value, and achieved power.\n Significance is not commercial meaning — compare to `mde` and\n a CPA / ROAS $ impact.\n- SRM → Invalid only on a randomized planned split (platform\n A/B / split-test, or an explicit planned allocation). Default\n one-ad-group, one-ad-per-variant delivery is auction-driven:\n do not run SRM as a gate; report imbalance as a caveat and\n compare rates, not totals. Broken setup (near-zero delivery,\n mid-test change, more than one variable) → Invalid. Low\n counts → Insufficient data. Many variants → Bonferroni.\n- Scale / kill / iterate needs a later yes\n (`omneky-launch-and-connect`).\n\n## Create\n\n1. Write the test plan with the user (one turn if they already\n gave it):\n - Hypothesis (what should win, and why)\n - Single variable (creative, or headline, or audience — pick one)\n - `primary_metric` (CTR or CVR — one only)\n - `baseline_rate` from `get_daily_metrics` with `date_range`,\n or a stated assumption\n - `mde` (absolute rate)\n - `alpha` 0.05 and `power` 0.8 unless they set others\n - `n_per_arm` from the two-proportion formula (show it;\n [ab-test-statistics](../_shared/references/ab-test-statistics.md))\n - `randomization_unit` and the caveat that platform ad-level\n splits are not true user randomization unless platform A/B\n is used\n - `attribution_window`\n - `horizon` (end date or sample). This is the stop rule.\n2. Variants come from the structured brief (or a prior creative\n handoff). Same brief except the one variable.\n3. Structure: one campaign, one ad group, one ad per variant\n (same targeting and budget split). Use an existing campaign\n only when they named it (`campaign_spec.id`).\n4. Preflight the channel with `omneky-launch-and-connect` rules:\n connect status, budget floor, targeting pre-call, copy / CTA /\n landing URL, credits. Default new launches to `PAUSED`.\n5. One summary: channel, tool, variants, budget split, floor,\n the full test plan, `PAUSED`. Wait for one yes.\n6. Call the matching `launch_*`. Then write the test plan back\n (`primary_metric`, `baseline_rate`, `mde`, `alpha`, `power`,\n `n_per_arm`, `randomization_unit`, `attribution_window`,\n `horizon`, campaign / ad ids).\n\nDo not go live unless that yes said to go live.\n\n## Measure\n\n1. Reuse the test plan. Missing ids: `get_campaigns` /\n `get_ad_groups` (pass `campaign_id`) / `get_ads`. Do not guess.\n2. Empty metrics for the brand + window + channel mean no imported\n data → stop. Empty is not zero.\n3. `get_performance_breakdown` (dimension ad or creative) and\n `get_daily_metrics` for the window. `get_performance_movers`\n when they asked what changed vs the prior period.\n4. Setup check on every look, including pre-horizon peeks.\n **Invalid** only for a broken setup: near-zero delivery\n on a variant, a variant changed mid-test, or more than\n one variable differs. Default setup (one ad group, one\n ad per variant) is auction-driven: do not run SRM as a\n gate. Report delivery imbalance as a caveat and compare\n rates, not totals. Run SRM → Invalid only when traffic\n was randomized with a planned split (platform A/B /\n split-test, or an explicit planned allocation) and\n χ² p < 0.001. If the setup check fails, give the verdict\n Invalid immediately, Stop, and recommend pausing or fixing the\n test (a pause is a later yes via\n `omneky-launch-and-connect`; never mutate here).\n5. Sample size: state impressions, clicks, and conversions per\n arm. Thin if either arm has fewer than about 1,000 impressions\n or 30 clicks (or 10 conversions when the metric is CVR) →\n Insufficient data. Do not declare a winner.\n6. Horizon not reached and setup check passed → **INTERIM**\n status only. No statistical verdict (Winner / Not\n commercially meaningful / Inconclusive / No detectable\n difference). Do not relabel a failed setup check as\n INTERIM. If they insist on early stopping, require an\n explicit acknowledgement and apply Bonferroni across\n looks or O'Brien-Fleming-style (see the statistics\n reference). If they will not acknowledge, say early\n stopping is not supported.\n7. Horizon reached (or corrected early stop): two-proportion\n z-test on the precommitted metric. Two-sided. Show n,\n successes, z, p-value, 95% CI, achieved power. This is a\n simple frequentist check, not an uncorrected sequential test\n — say that.\n8. Verdict:\n - **Invalid** — exempt from the horizon gate. From the\n step-4 setup check on every look, including peeks.\n Immediate Stop. Recommend pausing or fixing (later yes\n via `omneky-launch-and-connect`; never mutate here).\n - Only these wait for `horizon` (or a corrected early\n stop — lead with this):\n - **Inconclusive (underpowered)** — power < 0.8 and not\n significant\n - **No detectable difference** — |z| < 1.96 and power ≥ 0.8\n - Not commercially meaningful — significant but |lift| < `mde`\n (or $ impact below the stated floor)\n - **Winner** — significant, |lift| ≥ `mde`, $ impact stated\n9. Scale / kill / iterate is a later yes on\n `omneky-launch-and-connect`. Do not mutate here.\n\n## Failure and recovery\n\nAuth / 401 → `omneky-getting-started-and-failures` (host OAuth).\nEmpty reporting → empty is missing data, not zero; narrow the window.\n`credit_insufficient` → nothing charged; the user can add credits in the Omneky app.\n`needs_user_decision` → `request_user_decision`; never dump JSON.\nMissing tool → say so; Omneky app. Do not invent it.\nNever ask for internal ids. Never silent-switch `brand_id`.\n\n## Gaps\n\nThis connector cannot do comment/social inbox, landing pages,\nsubtitle burn-in, email/SMS, organic scheduling, Meta Ad Library\ncompetitor ads, virtual try-on, or website builder. Decline those.\nSee [capability-gaps](../_shared/references/capability-gaps.md).\nThere is no sequential-testing platform tool. Early stop is\nacknowledgement + correction, or not supported.\n"
"---\nname: omneky-ab-tests\ndescription: >-\n Create or measure an Omneky paid-media A/B test. Use for A/B\n test creation (one hypothesis, one variable, precommitted\n plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI,\n power, SRM, interim peek with no verdict, refuse a thin\n winner). Do not use for generic ROAS tables\n (omneky-analytics-and-catalogue), competitor analysis\n (omneky-competitor-analysis), first-run onboarding\n (omneky-onboarding), agency (omneky-agency), weekly marketing\n plans (omneky-marketing-plan), generating creatives\n (omneky-image-ads, omneky-product-video, omneky-avatar-ugc),\n or credit balance (omneky-billing). Keywords: A/B test,\n ab test create, ab test measurement, which variant won,\n significance, two-proportion z-test, MDE, SRM, interim peek,\n underpowered, commercially meaningful, budget split.\n---\n\n# A/B tests\n\nUse this when the user wants to set up an A/B test or read out which\nvariant won. One job: own the test plan and the readout.\n\nGate every tool on this session's `tools/list`. If a step's tool is\nmissing, say so and point to the Omneky app.\n\nPlan fields: [handoff-contract](../_shared/references/handoff-contract.md).\nFormulas: [ab-test-statistics](../_shared/references/ab-test-statistics.md).\nSourced ids: [preconditions](../_shared/references/preconditions.md).\nAcceptance gates: [creative-quality-gates](../_shared/references/creative-quality-gates.md).\nBrief: [creative-brief](../_shared/references/creative-brief.md).\nOther intents: [capability-registry](../_shared/references/capability-registry.md).\nWrite for: [marketer-persona](../_shared/references/marketer-persona.md).\nStep detail: [playbook](references/playbook.md) (Hard rules, Create, Measure).\n\n## Goal\n\nCreate a one-variable test with a precommitted plan (live or paused is\nthe user's choice in the host), or\nmeasure it with a two-proportion z-test and a marketer verdict.\n\n## When to use\n\n- Create an A/B test, measure one, \"which variant won\", significance,\n stop rule, interim peek, SRM, MDE\n\n## When not to use\n\n- Generic ROAS table → `omneky-analytics-and-catalogue`\n- Launch without a test plan → `omneky-launch-and-connect`\n\n## Prerequisites\n\n| Need | Check | If missing |\n|---|---|---|\n| Brand | `list_brands` | Ask which |\n| Channel | `list_connections` (`connector`) | Connect or managed |\n| Budget floor | `get_launch_requirements` | Do not launch under the floor |\n| Variants | prior creative handoff | Owning creative skill |\n| Baseline | `get_performance_summary` with `date_range` | State an assumed `baseline_rate` |\n| Coverage (measure) | `get_performance_summary` (`include_coverage=true`) | Do not declare a winner |\n| Metrics (measure) | `get_performance_breakdown` / `get_performance_summary` | Narrow the window |\n\n## Flow\n\n**Create**\n1. Write one hypothesis with one changed variable.\n2. Record metric, MDE, duration (`horizon`) and stop rule before launch.\n3. Budget at or above the floor. One ad group with one ad per variant\n does not guarantee an even spend split (delivery is auction-driven);\n say so. For an even split, use one ad group (or campaign) per variant\n with equal budgets, or the platform's A/B / split-test feature.\n4. Launch through `omneky-launch-and-connect`, live or paused as the\n user chose (the host confirms).\n\n**Measure**\n1. Confirm coverage for the window; empty is missing data, not zero.\n2. Run the setup check on every look (broken setup; SRM only for a\n randomized planned allocation, not default auction delivery). If it\n fails, give the verdict Invalid at once and recommend a pause.\n Never mutate here.\n3. Before `horizon`, report INTERIM with no statistical verdict.\n4. At `horizon`, run the z-test and give one verdict: Winner, Not\n commercially meaningful, Inconclusive, or No detectable difference.\n5. Refuse a winner when the sample is thin.\n\n## Failure and recovery\n\nAuth / 401 → `omneky-getting-started-and-failures` (host OAuth).\nEmpty reporting → `get_performance_summary` (`include_coverage=true`); empty is not zero.\n`credit_insufficient` → nothing charged; state balance vs required\n(`omneky-billing`). No payment links.\n`needs_user_decision` → fill the missing detail and call again; ask in plain chat only if you cannot. Never dump JSON.\nMissing tool → this session may run an older published app snapshot: look\nthe name up in [tool-name-fallbacks](../_shared/references/tool-name-fallbacks.md)\nand call the older name if that one is on `tools/list`. If neither is\nlisted, say so; Omneky app. Do not invent it.\nNever ask for internal ids. Never silent-switch `brand_id`.\n\n## Gaps\n\nDecline social inbox, landing pages, subtitle burn-in, email/SMS,\norganic scheduling, Ad Library ads, try-on, and website builder\n([capability-gaps](../_shared/references/capability-gaps.md)).\nThere is no sequential-testing platform tool. Early stop is\nacknowledgement + correction, or not supported.\n"
Full snapshot data
{
"description": "Create or measure an Omneky paid-media A/B test. Use for A/B test creation (one hypothesis, one variable, precommitted plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI, power, SRM, interim peek with no verdict, refuse a thin winner). Do not use for generic ROAS tables (omneky-analytics-and-catalogue), competitor analysis (omneky-competitor-analysis), first-run onboarding (omneky-onboarding), agency (omneky-agency), weekly marketing plans (omneky-marketing-plan), generating creatives (omneky-image-ads, omneky-product-video, omneky-avatar-ugc), or credit balance (omneky-billing). Keywords: A/B test, ab test create, ab test measurement, which variant won, significance, two-proportion z-test, MDE, SRM, interim peek, underpowered, commercially meaningful, budget split.",
"included_files": [
{
"relative_path": "references/playbook.md",
"size_in_bytes": 7149
}
],
"name": "omneky-ab-tests",
"skill_md_contents": "---\nname: omneky-ab-tests\ndescription: >-\n Create or measure an Omneky paid-media A/B test. Use for A/B\n test creation (one hypothesis, one variable, precommitted\n plan, launch live or paused as the user chooses) or A/B test measurement (z-test, CI,\n power, SRM, interim peek with no verdict, refuse a thin\n winner). Do not use for generic ROAS tables\n (omneky-analytics-and-catalogue), competitor analysis\n (omneky-competitor-analysis), first-run onboarding\n (omneky-onboarding), agency (omneky-agency), weekly marketing\n plans (omneky-marketing-plan), generating creatives\n (omneky-image-ads, omneky-product-video, omneky-avatar-ugc),\n or credit balance (omneky-billing). Keywords: A/B test,\n ab test create, ab test measurement, which variant won,\n significance, two-proportion z-test, MDE, SRM, interim peek,\n underpowered, commercially meaningful, budget split.\n---\n\n# A/B tests\n\nUse this when the user wants to set up an A/B test or read out which\nvariant won. One job: own the test plan and the readout.\n\nGate every tool on this session's `tools/list`. If a step's tool is\nmissing, say so and point to the Omneky app.\n\nPlan fields: [handoff-contract](../_shared/references/handoff-contract.md).\nFormulas: [ab-test-statistics](../_shared/references/ab-test-statistics.md).\nSourced ids: [preconditions](../_shared/references/preconditions.md).\nAcceptance gates: [creative-quality-gates](../_shared/references/creative-quality-gates.md).\nBrief: [creative-brief](../_shared/references/creative-brief.md).\nOther intents: [capability-registry](../_shared/references/capability-registry.md).\nWrite for: [marketer-persona](../_shared/references/marketer-persona.md).\nStep detail: [playbook](references/playbook.md) (Hard rules, Create, Measure).\n\n## Goal\n\nCreate a one-variable test with a precommitted plan (live or paused is\nthe user's choice in the host), or\nmeasure it with a two-proportion z-test and a marketer verdict.\n\n## When to use\n\n- Create an A/B test, measure one, \"which variant won\", significance,\n stop rule, interim peek, SRM, MDE\n\n## When not to use\n\n- Generic ROAS table → `omneky-analytics-and-catalogue`\n- Launch without a test plan → `omneky-launch-and-connect`\n\n## Prerequisites\n\n| Need | Check | If missing |\n|---|---|---|\n| Brand | `list_brands` | Ask which |\n| Channel | `list_connections` (`connector`) | Connect or managed |\n| Budget floor | `get_launch_requirements` | Do not launch under the floor |\n| Variants | prior creative handoff | Owning creative skill |\n| Baseline | `get_performance_summary` with `date_range` | State an assumed `baseline_rate` |\n| Coverage (measure) | `get_performance_summary` (`include_coverage=true`) | Do not declare a winner |\n| Metrics (measure) | `get_performance_breakdown` / `get_performance_summary` | Narrow the window |\n\n## Flow\n\n**Create**\n1. Write one hypothesis with one changed variable.\n2. Record metric, MDE, duration (`horizon`) and stop rule before launch.\n3. Budget at or above the floor. One ad group with one ad per variant\n does not guarantee an even spend split (delivery is auction-driven);\n say so. For an even split, use one ad group (or campaign) per variant\n with equal budgets, or the platform's A/B / split-test feature.\n4. Launch through `omneky-launch-and-connect`, live or paused as the\n user chose (the host confirms).\n\n**Measure**\n1. Confirm coverage for the window; empty is missing data, not zero.\n2. Run the setup check on every look (broken setup; SRM only for a\n randomized planned allocation, not default auction delivery). If it\n fails, give the verdict Invalid at once and recommend a pause.\n Never mutate here.\n3. Before `horizon`, report INTERIM with no statistical verdict.\n4. At `horizon`, run the z-test and give one verdict: Winner, Not\n commercially meaningful, Inconclusive, or No detectable difference.\n5. Refuse a winner when the sample is thin.\n\n## Failure and recovery\n\nAuth / 401 → `omneky-getting-started-and-failures` (host OAuth).\nEmpty reporting → `get_performance_summary` (`include_coverage=true`); empty is not zero.\n`credit_insufficient` → nothing charged; state balance vs required\n(`omneky-billing`). No payment links.\n`needs_user_decision` → fill the missing detail and call again; ask in plain chat only if you cannot. Never dump JSON.\nMissing tool → this session may run an older published app snapshot: look\nthe name up in [tool-name-fallbacks](../_shared/references/tool-name-fallbacks.md)\nand call the older name if that one is on `tools/list`. If neither is\nlisted, say so; Omneky app. Do not invent it.\nNever ask for internal ids. Never silent-switch `brand_id`.\n\n## Gaps\n\nDecline social inbox, landing pages, subtitle burn-in, email/SMS,\norganic scheduling, Ad Library ads, try-on, and website builder\n([capability-gaps](../_shared/references/capability-gaps.md)).\nThere is no sequential-testing platform tool. Early stop is\nacknowledgement + correction, or not supported.\n"
}SHA-256 of public snapshot: 98fbd2470c67614f02944540754b9bc2813e76da7627c4ad065cd2a3a21bae89