Fullstory

View saved version →

---
name: comparisons
description: How to structure A vs B comparisons in Fullstory — when to use dimensionality (event/session properties) vs separate segments (user-level properties), and why the distinction matters for correctness.
user-invocable: false
---

# Comparisons

When the user asks to compare A vs B, the right mechanism depends on what the comparison axis is. Use the decision table below to classify it — the user doesn't need to know this distinction exists.

## Event/session properties → dimensionality

If the comparison axis describes the context of an individual event at the moment it fired — not the user who triggered it — use dimensionality. Common examples: device type, browser, OS, page URL, element. But the rule is the principle, not the list: if the property travels with the event, not the user, it belongs here. Express it as a single `top_n` metric with the comparison axis as the grouping dimension.

Example: "rage clicks on mobile vs desktop" → `fullstory:build_metric(query="rage clicks by device type", output_type="top_n")`. The result table shows mobile and desktop as separate rows.

To refine an established comparison metric (e.g., "add a Chrome-only filter"), pass its `metric_id` to `fullstory:update_metric` with a refinement instruction rather than rebuilding.

**Do not use segments for event properties.** Building a "mobile users" segment and a "desktop users" segment would assign all of a user's rage clicks to whichever device they ever used — even clicks that happened on the other device.

## User-level properties → separate segments

Properties that describe a user rather than an event should use segments. The key mechanism: Fullstory resolves user properties to the user's **last known value** for that key. This canonical value is what segment queries match against — so you're asking "what bucket is this user in now?", not "what was their value at the moment of each event?".

Built-in user properties that work this way: `signed_up` (signed-up status), `first_seen` / `last_seen` (dates), `total_sessions` (engagement depth), and any custom user properties (`user_var_string`, `user_var_int`, etc.) set via `setUserProperties` — e.g. plan type or account ID. Build one metric and one segment per cohort, then compute each cohort in sequence: attach the segment via `fullstory:update_metric(metric_id, segment_id)`, call `fullstory:compute_metric(metric_id)`, store the result, then repeat with the next segment. Present the results side by side. Do not pass `segment_id` directly to `fullstory:compute_metric`.

Example: "do enterprise users experience more errors than free users?" → build two segments (enterprise, free), build one metric (errors), compute twice.

To refine a cohort after it's been built (e.g., "also exclude trial users from the free segment"), use `fullstory:update_segment` with the existing `segment_definition` rather than rebuilding with `fullstory:build_segment`.

Using `top_n` dimensionality for user properties is valid if you specifically want point-in-time values — each event is attributed to the user property value at the moment it fired. If a user changed plan tier mid-period, their events will be split across both values. For most comparisons you want the canonical (current) value, which is why segments are the default choice.

## Decision table

| Comparison axis | Type | Mechanism |
|----------------|------|-----------|
| Device type, browser, OS | Event property | Dimensionality |
| Page URL, element | Event property | Dimensionality |
| `signed_up`, `first_seen`, `last_seen` | User property | Segments |
| `total_sessions` | User property | Segments |
| `user_var_*` (custom user properties) | User property | Segments |

If you can't tell whether a property is event-level or user-level, default to dimensionality — it's more precise and uses fewer API calls.

## Why the wrong choice produces wrong results

**Segments for event properties (the temporal scope problem):** A segment matches users by their canonical properties, then includes all of that user's events. Alice uses both mobile and desktop during a 30-day window. She rage-clicks 5 times — all on desktop. With a "mobile users" segment, Alice qualifies (she used mobile once), so all 5 desktop rage clicks inflate the mobile count. With a device-type dimension, each rage click is tagged with the device it actually fired on — all 5 go to desktop, zero to mobile.

**Dimensionality for user properties (the split-value problem):** Bob was on the free plan for two weeks, then upgraded to enterprise. Using `top_n` grouped by plan tier, his events split — two weeks of errors under "free", two weeks under "enterprise". If the question was "do enterprise users see more errors?", Bob's pre-upgrade errors are excluded from the enterprise count. With segments, Bob's canonical value is "enterprise" (last known), so all his events count toward the enterprise cohort.

View saved version →

---
name: general-analysis
description: Fullstory analytics workflow. Use when answering a question that requires measuring user behavior — counts, rates, trends, breakdowns, or cohort comparisons. Builds segments and metrics, computes results, then investigates sessions to explain what the numbers mean.
---

# Fullstory Analytics

## Mental Model

Internalize these three concepts before choosing tools:

- **Segment** = a cohort of users (the "who"). A segment is a filter, not a measurement. It narrows which users' data a metric runs against.
- **Metric** = the measurement (the "what" and "how much"). Every quantitative answer is a metric. Even "how many users visited /checkout" is a metric (count of page views), optionally filtered by a segment.
- **Session** = evidence (the "why"). Sessions are qualitative. Use them to understand *why* a number looks the way it does — not to answer the quantitative question itself.

## Step 0: Classify Intent

Before calling any tool, determine what the user is asking for:

- "how many", "what's the count", "what percentage", "what's the rate" → quantitative answer → `single_number` metric
- "which pages", "top N", "by browser", "breakdown by" → breakdown → `top_n` metric
- "over time", "by day", "is it getting worse", "trend" → trend → `trend` metric
- "mobile vs desktop", "compare", "A vs B" → comparison → invoke the `comparisons` skill
- "show me sessions", "let me watch", "examples of" → session exploration → `fullstory:get_sessions` with `metric_id`
- "sessions from power users", "show me what enterprise users do" → cohort browsing → `fullstory:build_segment` then `fullstory:get_sessions` with `segment_id`

If the intent is ambiguous, ask the user before proceeding. Getting the intent wrong wastes a build+compute cycle.

## Step 1: Resolve or Build

### Always search before building

Users often don't know what metrics or segments already exist in their Fullstory account. Always search first, even when the question sounds ad-hoc. Use `fullstory:get_metric(regex="...")` or `fullstory:get_segment(regex="...")`, starting broad and narrowing if needed (e.g., "how many rage clicks on checkout?" → start with `checkout`, then try `checkout.*rage` if the first search returns too many results).

Results include a short description of the segment's filters and events, so use that — not just the name — to judge relevance. If no results match, tell the user nothing was found and confirm before building. If results come back but their filters/events don't match the question, tell the user what you found and that none seem to match, then confirm they'd like you to build a new one.

If 2 or more plausible candidates come back, immediately call `fullstory:get_view_counts` on their IDs (up to 10) to rank by popularity. If search returns more than 10 candidates, pass the 10 most name-similar IDs. Then:

- If one candidate has clearly more views (roughly 5x or more than the next), treat it as the canonical object — proceed with it and tell the user you're using "the most-used version."
- If the top 2–3 are comparable in view count, present them sorted by popularity. Use the filters, events, and description fields from the search results to explain what each one measures or captures differently, then ask the user which to use.
- If all candidates have zero or near-zero views, flag them as likely stale and offer to build fresh.

### Building new

**Metrics:** Before building, make sure the unit of measurement is correct — getting this wrong is the most common source of misleading results. If the question is about "customers", "accounts", or "organizations", clarify whether the user wants to count individual users or group users by a customer/account/organization property. If it's the latter, look for user properties that match and build the metric to count by that property. Similarly, watch for ambiguity between pages and URLs — "which pages" usually means page titles or paths, not full URLs with query parameters.

Call `fullstory:build_metric` with a descriptive query and the correct `output_type` derived from intent classification:

- Quantitative answer → `single_number`
- Breakdown → `top_n`
- Trend → `trend`

For `top_n`, make sure the grouping dimension is expressed in the query (e.g., "top pages by rage click count"). The metric builder will not invent a dimension on its own.

**Segments:** Call `fullstory:build_segment`. Always reference by `segment_id` in subsequent steps. If the same cohort is needed for multiple questions in the conversation, reuse the existing `segment_id` — do not rebuild.

### Refining existing

If the user wants to modify a metric or segment already established in this conversation — adding or removing a filter, changing aggregation, adjusting the time range, or changing output shape — use `fullstory:update_metric` or `fullstory:update_segment`. Pass the existing `metric_id` or `segment_definition` and a natural language `refinement`.

- `fullstory:update_metric`: accepts `metric_id` and supports two mutually exclusive modes: LLM refinement (filter changes, aggregation changes, output type overrides via `output_type`) and segment attachment (attach a `segment_id` to the metric). Does not support ratio metrics — rebuild those with `fullstory:build_metric`.
- `fullstory:update_segment`: supports filter additions/removals and time range changes.

## Step 2: Compute

Call `fullstory:compute_metric` with:
- `metric_id` — the ID returned by `fullstory:build_metric`, `fullstory:get_metric`, or `fullstory:update_metric`
- `time_range` — default is `last_30_days`; ask the user if they want a different window

If the question is scoped to a cohort, segments must be pre-attached before computing. Call `fullstory:update_metric(metric_id, segment_id)` first, then call `fullstory:compute_metric(metric_id)`. Do not pass `segment_id` directly to `fullstory:compute_metric`.

Present results in plain language with context:
- Numbers: "12,340 dead clicks over the last 30 days"
- Tables: highlight the top entries; include percentages if a total is available
- Trends: call out direction, magnitude, and any inflection points

Always surface `metric_url` so the user can verify in the Fullstory UI.

## References

Load these when the situation calls for it:

- `references/validation.md` — when results are zero, anomalous, or the user expresses skepticism
- `references/sessions.md` — when investigating sessions to understand why a metric looks the way it does

## Guidelines

- Default `time_range` is `last_30_days`. Ask before using a different window unless the user specified one.
- When building a segment for use in a later step, always reference by `segment_id`.
- Reuse `segment_id` and `metric_id` within a conversation. Do not rebuild objects the user has already established.
- If the user asks for a different shape of an existing metric (e.g., they have a count but now want a trend), call `fullstory:update_metric` with the existing `metric_id` and the desired `output_type`. Only fall back to `fullstory:build_metric` for fundamentally different queries or ratio metrics.
- When presenting table results, include both the dimension value and the count. If a total is available, show percentages.
- Always surface `metric_url` in your response so the user can verify in the Fullstory UI. `fullstory:build_metric` and `fullstory:update_metric` both return `metric_url` — surface it as soon as it's available, don't wait until after computing.

Referenced files: 3

View saved version →

---
name: session-review
description: Use when diagnosing user-reported issues, investigating bugs, analyzing user behavior, or validating UI correctness using Fullstory session recordings.
---

# Session Review

Review Fullstory sessions to understand what happened — whether diagnosing a customer-reported bug or validating UI changes during development.

## When to Use

- User reports a bug and provides a session URL
- Investigating an error or unexpected behavior
- Understanding user flow through the application
- Debugging issues that are hard to reproduce
- Validating UI changes locally after testing

## MCP Tools

The tools follow this pattern:

```
fullstory:session_open  →  (fullstory:session_screenshot | fullstory:session_get_a11y_tree | fullstory:session_diff)*  →  fullstory:session_close
```

1. **`fullstory:session_open`**: Pass `session_id`. Returns event summaries and a `client_id`. The `session_id` accepts a bare id (`<device-id>:<session-id>`), a full session URL, or a short link.
2. **`fullstory:session_screenshot`**: Rendered screenshot at a timestamp. Pass `client_id`, `page_id`, `timestamp` (ms from session start); optional `full_page` for the whole scrollable page. Sequential calls with increasing timestamps are faster.
3. **`fullstory:session_get_a11y_tree`**: Page structure (accessibility tree / DOM) at a timestamp. Pass `client_id`, `page_id`, `timestamp`.
4. **`fullstory:session_diff`**: Highlights changes between two timestamps within a page. Pass `client_id`, `page_id`, `from_ts`, `to_ts`.
5. **`fullstory:session_close`**: Always call when done to free resources. Pass `client_id`.

`session_view` is deprecated and removed — it was split into `session_screenshot` (the rendered image) and `session_get_a11y_tree` (page structure). Use those two instead.

## Workflow

### 1. Open the Session
```
fullstory:session_open(session_id="https://app.fullstory.com/ui/<org-id>/session/<device-id>:<session-id>")
```
Returns: event summaries (navigation, clicks, errors, custom events) and `client_id`. You can also pass a bare `<device-id>:<session-id>` id or a short link as `session_id`.

### 2. Identify Key Moments
Scan event summaries for:
- **Page navigations** — entry points into different views
- **Click events** — user interactions with buttons, links, forms
- **Error events** — console errors, network failures, renderer errors
- **Rage clicks / mouse thrash** — signs of UX friction
- **Custom events** — application-specific telemetry

Note the `page_id` and `timestamp` for each moment you want to inspect.

### 3. Visual Inspection
At each key moment, capture the screenshot — and the accessibility tree when you need the structure or text content:
```
fullstory:session_screenshot(client_id="<id>", page_id="<page>", timestamp=<ms>)
fullstory:session_get_a11y_tree(client_id="<id>", page_id="<page>", timestamp=<ms>)
```
Check for:
- Layout correctness (elements positioned properly)
- Content rendering (text, images, icons present)
- Responsive behavior (no overflow, proper sizing)
- Empty/loading states handled

### 4. Compare States
For before/after analysis:
```
fullstory:session_diff(client_id="<id>", page_id="<page>", from_ts=<before_ms>, to_ts=<after_ms>)
```
Returns a screenshot with changed regions highlighted plus a text summary of component changes.

### 5. Report Findings
Summarize what was observed:
- What the user saw at each key moment
- Any errors, visual regressions, or unexpected behavior
- Root cause analysis if diagnosing a bug
- Confirmation of correctness if validating changes

### 6. Cleanup
Always close the session:
```
fullstory:session_close(client_id="<id>")
```

## Example

When a user reports "the checkout button didn't work" and provides a session URL:

1. Call `fullstory:session_open` with the session URL as `session_id`
2. Find the checkout-related events in the summaries (note the `page_id` and `timestamp`)
3. Use `fullstory:session_screenshot` — and `fullstory:session_get_a11y_tree` if you need the DOM — to see the UI state just before the button click
4. Use `fullstory:session_diff` to compare before/after the click attempt to see what changed (or didn't)
5. Report findings: what the user saw, any errors, why the action may have failed
6. Call `fullstory:session_close` to clean up

## Common Mistakes

| Mistake | Fix |
|---------|-----|
| Forgetting to close session | Always call `fullstory:session_close` — even on errors |
| Using `session_view` | Removed — use `fullstory:session_screenshot` + `fullstory:session_get_a11y_tree` |
| Random timestamp access | Use sequential increasing timestamps for faster access |
| Skipping the diff tool | `fullstory:session_diff` highlights changes automatically — faster than comparing two screenshots manually |
| Not checking for error events | Scan event summaries for errors before visual inspection |