evals/RUBRIC.md

# Marketing Council Evaluation Rubric

Score each dimension from 0 to 2.

## 1. Diagnosis

- 0: jumps to tactics or repeats the request
- 1: names a plausible problem but does not isolate the dominant constraint
- 2: identifies the decision and separates symptoms from causes

## 2. Evidence

- 0: invents market/customer facts or hides source quality
- 1: mentions evidence but mixes it with assumptions
- 2: distinguishes fact, evidence, inference, assumption, hypothesis, and unknown where material

## 3. Freshness

- 0: relies on stale or unverified current claims
- 1: notes uncertainty but does not resolve load-bearing current facts
- 2: verifies freshness-sensitive claims when tools are available or labels them unknown

## 4. Audience and buying situation

- 0: uses generic personas or demographic stereotypes
- 1: identifies a plausible audience
- 2: ties audience choice to buying triggers, behavior, reachability, or roles

## 5. Economics

- 0: ignores economics when they could reverse the decision
- 1: mentions cost or revenue without decision logic
- 2: uses price, margin, CAC, LTV, payback, retention, capacity, or contribution where relevant

## 6. Separate strategy from tactics

- 0: tactic inventory is presented as strategy
- 1: strategic direction exists but exclusions are weak
- 2: one primary strategic choice, meaningful exclusions, then tactics that execute or test it

## 7. Trade-offs and disagreement

- 0: presents one view as universally correct
- 1: mentions alternatives
- 2: explains credible disagreement and resolves it using deciding factors

## 8. Measurement

- 0: vanity metrics or no decision thresholds
- 1: relevant metrics but no stop/success rule
- 2: business outcome, leading indicators, guardrails, success threshold, failure threshold, and next decision

## 9. Uncertainty

- 0: false certainty
- 1: vague caveats
- 2: visible assumptions, confidence appropriate to evidence, and evidence that would reverse the recommendation

## 10. Case sensitivity

- 0: answer could be reused unchanged across unrelated cases
- 1: some adaptation
- 2: market, awareness, economics, buying behavior, channel, or product conditions materially change the recommendation

A strong run normally scores 16 or more, with no zero on diagnosis, evidence, economics when relevant, measurement, or uncertainty.