skills/creative-testing/references/experiment-design.md

# Creative Experiment Design

## Test card

Document before production:

- Business decision
- Objective and audience/context
- Current control
- Primary variable
- Invariants
- Hypothesis and mechanism
- Variants
- Primary metric
- Guardrail metrics
- Placement/account and run window
- Allocation or delivery assumptions
- Minimum practical effect worth acting on
- Decision rule
- Known validity risks

## Variable discipline

Good single variables include opening hook, benefit frame, proof mechanism, creative format, visual treatment, CTA framing, or creator delivery. If multiple elements must change together, name the test as a concept/territory test and do not claim which element caused the result.

## Metrics by job

- Attention: qualified reach, early video retention, thumb-stop/hold proxy where supplied
- Utility or resonance: saves, shares, substantive comments, completion where appropriate
- Consideration: profile actions, link clicks, landing-page engagement where available
- Response: leads, bookings, qualified inquiries, orders, or sales from a connected measurement source
- Audience cost: hides, unfollows, negative comments, low-quality leads

Simplified social analytics may not provide every business outcome. State that limitation and avoid substituting engagement for revenue.

## Interpretation

Check unequal reach, different publication time, audience mix, placement, paid support, creative format, and missing conversion data. Organic results are generally directional. Report magnitude and uncertainty; do not use “statistically significant” without exposure-level data and a valid analysis.

## Learning record

Write: result, evidence, limitations, decision, scope, and next test. A durable learning is specific enough to guide future creative but narrow enough not to overgeneralize.