← TolstoyCONTENT HISTORY

Update to Tolstoy

Snapshot Nov 17, 2026 · 12:10 UTC · version 1.4.1

Collection source: downloaded plugin package.

WHAT CHANGED

Instructions updated for ecommerce-analytics

Skill instructions — Now says: The report leaves out events sent from a page whose address contains `localhost`, `127.0.0.1` or `shopifypreview.com`, or starts with `file`, and events sent...

Observed in instructions or declared skills. Runtime behavior has not been tested.

Save Tolstoy and check future changes in Following.

Up to 3 plugins free · no card required.

All fields and original values.

Skill instructions

Before

Nov 16, 2026 · 18:12 UTC

No removed lines (ordering may have changed)

After

Nov 17, 2026 · 12:10 UTC

The report leaves out events sent from a page whose address contains `localhost`, `127.0.0.1` or `shopifypreview.com`, or starts with `file`, and events sent by headless Chrome or Lightpanda (a user agent that contains `HeadlessChrome` o...

Compare saved observations

Download comparison JSON

SKILL.md text changes

− removed · + added

--- before
+++ after
@@ -267,6 +267,15 @@
    `tolstoy analytics get-store-analytics --sections '["details"]' --json`
    with `--date-from` and `--date-to`.
 
+The report leaves out events sent from a page whose address contains
+`localhost`, `127.0.0.1` or `shopifypreview.com`, or starts with `file`, and
+events sent by headless Chrome or Lightpanda (a user agent that contains
+`HeadlessChrome` or `Lightpanda`) or by some crawlers and audit tools, such as
+Googlebot and Lighthouse. Tolstoy still answers them with HTTP 200. A test on
+your own machine, or in a headless browser such as Playwright in its default
+mode, checks delivery only. To check the report, send the events from the
+deployed store in a normal browser.
+
 Then note the events and their rules in the project's `AGENTS.md`, so that the
 next agent reads the numbers the same way.
 
Full technical diff · 1 changed fields

changed /skill_md_contents

BEFORE
"---\nname: ecommerce-analytics\ndescription: Plan, add and judge analytics for an online store - what to measure and why, which storefront events to send and what each one means, how to work out rates from event and shopper counts, how to judge a number against per-vertical benchmarks from Tolstoy's store data, and how to add it free with the Tolstoy SDK and read it with the Tolstoy CLI or MCP. Use when asked to add analytics or tracking to a store, to measure a store feature, page or change, to set up funnel or conversion tracking, to choose e-commerce metrics, or to judge numbers such as average order value or refund rate, even when Tolstoy is not named. To read a Tolstoy report that already exists, use store-analytics.\nmetadata:\n  author: Tolstoy\n  version: \"1.0.0\"\n---\n\n# E-commerce analytics\n\nE-commerce analytics tells a store which shopper actions lead to orders, and\nwhether a change made them go up. Measure a few things that decide something,\ngive each event a fixed meaning, count through to the order, and judge each\nnumber against a baseline.\n\n## Principles\n\n1. **Start from the decision.** Before you add an event, write the question it\n   answers and what the store will do with the answer. An event that decides\n   nothing is noise in the report.\n2. **Revenue = sessions × conversion rate × average order value**, with the\n   conversion rate as orders ÷ sessions. A feature can raise revenue only\n   through one of the three. Name the one that it should move, and measure that\n   one.\n3. **Count shoppers, not events.** One shopper can send an event many times.\n   Rates divide shoppers by shoppers, never raw event counts.\n4. **Count through to the order.** Views and clicks measure interest. The share\n   of shoppers who go on to order measures value, so send an event at the order\n   too.\n5. **Compare like with like.** Compare periods of the same length, with the same\n   weekdays, and no sale or launch in only one of them.\n6. **Users of a feature are not proof that it works.** Shoppers who choose a\n   feature often were more likely to buy anyway, so a higher rate for them is a\n   correlation. For proof, show the feature to a random half of shoppers and\n   compare the two halves.\n7. **Judge the difference, not each rate.** For two rates p₁ of n₁ shoppers\n   and p₂ of n₂ shoppers, the 95% range of the difference is\n   (p₂ − p₁) ± 1.96 × √(p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂). 30 buyers of 1,000\n   shoppers (3.0%) against 42 of 1,000 (4.2%) is a difference of 1.2 points\n   ± 1.6, so it can be zero. Report it as an estimate, not as a proven effect,\n   and collect more shoppers.\n\n## What to measure\n\n| Metric | Formula | It answers |\n|---|---|---|\n| Orders, revenue | Count and sum of the orders in the period | Does the store grow? |\n| Average order value (AOV) | Revenue ÷ orders | Do shoppers spend more per order? |\n| Conversion rate | Orders ÷ sessions | Do visits turn into orders? |\n| Buyer rate | Shoppers who ordered ÷ shoppers who visited | Do visitors become buyers? |\n| Step rate | Shoppers at a step ÷ shoppers at the step before | Where do shoppers drop out? |\n| Feature buyer rate | Shoppers who ordered after the feature ÷ shoppers who used it | Does the feature lead to orders? |\n| Refund rate | Refunded orders ÷ orders | Does the revenue stay? |\n\nSay for each revenue number whether it includes discounts, shipping and tax.\n\n## Events to send\n\n| Event | Send it when |\n|---|---|\n| `page_view` | Any page of the store shows. |\n| `product_view` | A product page shows. |\n| `add_to_cart` | The cart accepts an item. |\n| `checkout_started` | The shopper opens checkout. |\n| `order_completed` | The order confirmation page shows, once per order. |\n| `<feature>_view` | The feature shows to the shopper, for example `size_guide_view`. |\n| `<feature>_open` | The shopper opens or uses the feature, for example `size_guide_open`. |\n| `order_after_<feature>` | The order confirmation page shows within your window after `<feature>_open` (see the window rule below). |\n\n- Send `order_completed` from a page where the store's own code runs. A hosted\n  checkout that runs no store code cannot send it; there, take orders and\n  buyers from the store platform.\n- Put details, such as the product ID or the order ID, in the parameters.\n  Tolstoy keeps them, but the report does not sum or group them. A value that\n  you must count apart, such as the half of a test, goes in the event name:\n  `size_guide_test_a`, `size_guide_test_b`.\n\n## Give each event a meaning\n\nTolstoy's report gives its own widget events a meaning with fixed rules. A set\nof event names counts as an open or as an add to cart. An interaction counts\nonly when the shopper acted: autoplay does not. An order counts for a widget\nwhen the same shopper placed it within a window after an open, 24 hours by\ndefault, with both in the report's period. Tolstoy does not know what your\nevent names mean, so you set the same kind of rules for them, and keep them in\nthe code:\n\n1. **One meaning per name.** Write down each event name, what it means, and the\n   rate it feeds, before you send it. Use one lower-case name for one action\n   on every page, and never rename it: a new name starts a new count.\n2. **Only deliberate acts are actions.** Send an action event, such as\n   `size_guide_open`, on a click, a tap or a submit. A page load, autoplay,\n   scrolling past and closing are not actions. Send what shows on its own as\n   a `_view` event.\n3. **Only on success, and once.** Send `add_to_cart` after the cart accepts the\n   item, not on a click that can fail. Send `order_completed` once per order:\n   keep the order ID in `localStorage` and skip the event when the page\n   reloads.\n4. **Apply the window rule in the store's code.** The report cannot tell\n   whether one shopper sent two events. To credit an order to an action, keep\n   the time of the action in `localStorage`, and at the order send a second\n   event when the action came within the window. Choose the window before you\n   read any numbers; 24 hours matches Tolstoy's report. Store and use the time\n   only when the shopper allowed analytics: check the `tolstoy-cookie-policy`\n   value that the store sets from its consent tool for the SDK.\n\n   ```javascript\n   const analyticsAllowed = () =>\n     localStorage.getItem('tolstoy-cookie-policy') !== 'rejected';\n\n   // When the shopper opens the size guide:\n   if (analyticsAllowed()) {\n     localStorage.setItem('size_guide_opened_at', String(Date.now()));\n   }\n   tolstoy.analytics.track('size_guide_open');\n\n   // On the order confirmation page, once per order:\n   const openedAt = Number(localStorage.getItem('size_guide_opened_at'));\n   if (analyticsAllowed() && openedAt && Date.now() - openedAt <= 24 * 60 * 60 * 1000) {\n     tolstoy.analytics.track('order_after_size_guide', { orderId });\n   }\n   ```\n\n5. **Start the rate where the rule starts.** Divide the shoppers of\n   `order_after_size_guide` by the shoppers of `size_guide_open`, not by all\n   visitors.\n6. **Rows do not add up.** One order can send several `order_after_<feature>`\n   events, so do not add them up across features.\n\n## Work out the numbers\n\nRead the report from your own event plan. Each `tolstoy-sdk.<name>` row means\nwhat your plan says: the action, the pages that send it, and its rule. For each\nevent name, the report gives two counts for the period: `events`, how many\ntimes the event was sent, and `shoppers`, how many different shoppers sent it.\nYou compute every rate from these counts.\n\n| You want | Compute it from one report |\n|---|---|\n| Orders | `events` of `order_completed` |\n| Buyers | `shoppers` of `order_completed` |\n| Orders per buyer | `events` ÷ `shoppers` of `order_completed` |\n| Buyer rate | `shoppers` of `order_completed` ÷ `shoppers` of `page_view` |\n| Step rate | `shoppers` of a step ÷ `shoppers` of the step before, such as `add_to_cart` ÷ `product_view` |\n| Buyers after a feature | `shoppers` of `order_after_<feature>` |\n| Feature buyer rate | `shoppers` of `order_after_<feature>` ÷ `shoppers` of `<feature>_open` |\n\n- Ask for the whole period in one report. A shopper is counted once per\n  report, so the `shoppers` of separate reports, such as one per day, do not\n  add up.\n- The report counts each name on its own within the period. A shopper who\n  opened the size guide the day before the period and ordered within it adds\n  to `order_after_size_guide` but not to `size_guide_open`. So each rate is a\n  ratio of two counts, not the share of one group of shoppers. Ask for a period\n  much longer than the time from the first event to the second, such as 28 days\n  for a 24-hour window, so that these edges stay small.\n- The SDK keeps one shopper ID per browser and sends nothing for a shopper who\n  rejected analytics. Each count covers the browsers that allowed analytics,\n  and one person on two devices counts twice. A ratio of two events shares\n  these limits on both sides.\n\nWhat the counts cannot show:\n\n- **A strict funnel.** A ratio of two events does not check that the same\n  shoppers did both within the period: a buyer can skip a step, come back in\n  another browser, or have done the first step before the period. The window\n  rule only makes sure that each `order_after_<feature>` comes from a browser\n  that opened the feature, maybe before the period.\n- **Revenue and AOV.** The report does not sum the order values that you send\n  as parameters. Take revenue from the store's orders.\n- **Today.** The report covers whole UTC days before today. Events of today\n  show from about 06:30 UTC the next day.\n\n## Measure a feature\n\n1. Write the question and its metric: \"Do shoppers who see the size guide buy\n   more?\" is the buyer rate per half.\n2. Fix the test dates and the window before the test starts, for example 14\n   days and 24 hours.\n3. On the page of the feature, within the test dates, put each new shopper in\n   half `a` or `b` at random, keep the half in `localStorage`, and show the\n   feature to half `a` only. When analytics is allowed, also keep the time,\n   and send `size_guide_test_a` or `size_guide_test_b`.\n4. At the order, when a time is kept and the order comes within the window\n   after it, send `order_after_size_guide_test_a` or\n   `order_after_size_guide_test_b`, once per order.\n5. Read one report from the first test day to the last test day plus the\n   window, so it holds every shopper's start and full window. For each half,\n   divide the `shoppers` of its order event by the `shoppers` of its test\n   event: every buyer is one of the shoppers of the half, and each had the same\n   window to order. The difference between the halves estimates the effect of\n   the feature. It is an effect only when its 95% range (principle 7) does not\n   include zero.\n\nWithout a split, the feature buyer rate and a comparison with the weeks before\nthe launch are correlations. Report them as such.\n\n## Judge a number\n\n1. Check the definition: the period, the time zone, the currency, what revenue\n   includes, and the rules of your events.\n2. Check the base: `0` is a measured zero. An event name that is missing from\n   the report was not sent in the period, or is not among the 120 most\n   frequent names. Compute the range of the difference (principle 7) before you\n   call a change real.\n3. Compare with the store's own previous period of the same length. The\n   store's own trend beats any benchmark.\n4. Then compare with the benchmarks of the store's vertical. A number outside\n   the middle half is worth a look, not a verdict: the price of the products\n   decides most of the AOV.\n\n## Benchmarks from Tolstoy stores\n\nMedians across stores that send Tolstoy their orders, read from Tolstoy's\nwarehouse on 2026-11-16.\n\n| Vertical | Median AOV | Middle half of stores | Stores | Median refund rate | Middle half of stores | Stores |\n|---|---|---|---|---|---|---|\n| Beauty & Wellness | $62 | $42–$105 | 621 | 2.2% | 1.3–3.9% | 575 |\n| Fashion | $122 | $76–$181 | 332 | 5.8% | 2.8–11.8% | 319 |\n| Accessories | $87 | $54–$160 | 154 | 3.8% | 1.7–6.7% | 136 |\n| Home | $217 | $95–$582 | 120 | 4.3% | 2.5–6.4% | 109 |\n| Food & Beverages | $60 | $44–$101 | 112 | 2.1% | 1.2–3.8% | 102 |\n| Health | $69 | $44–$94 | 84 | 2.6% | 1.5–4.3% | 77 |\n| Sports & Outdoors | $129 | $61–$196 | 57 | 2.7% | 1.9–5.2% | 55 |\n| Baby & Kids | $89 | $60–$168 | 51 | 3.4% | 2.1–5.2% | 51 |\n| Pets | $61 | $33–$140 | 30 | - | - | - |\n\n- **Stores:** each store once, with all its orders in USD, at least 30 orders\n  in the window, and a vertical from Tolstoy's classification of the store\n  (stores that joined after April 2026 have none). A vertical with fewer than\n  30 stores is left out. The middle half runs from the 25th to the 75th\n  percentile of stores.\n- **AOV:** the store's revenue ÷ its orders, for orders from 2026-08-18 to\n  2026-11-15 (90 UTC days). Revenue is the order's items after discounts,\n  without shipping and tax.\n- **Refund rate:** the share of a store's orders with a refund above zero, in\n  part or in full. Orders from 2026-08-18 to 2026-10-16, refunds up to\n  2026-11-15, so each order had 30 to 89 days for a refund. Stores with no\n  refund in the window (88 of 1,512) are left out, because Tolstoy may not\n  receive their refunds.\n- **No conversion-rate benchmark:** Tolstoy does not see a store's total\n  visits, so it cannot compute a store's conversion rate.\n\n## Add it with Tolstoy\n\nSending events and reading the report are free on every Tolstoy plan, with no\nTolstoy widget on the store. Read\nhttps://developers.gotolstoy.com/agents/ecommerce-analytics.md before you write\ncode: it has each step, the SDK code, the shopper's consent, the same-day\ndelivery check, and what the report does not give.\n\n**With the Tolstoy tools connected** (`list_workspaces`, `get_store_analytics`):\n\n1. Take the app key from `currentAppKey` of `list_workspaces`.\n2. Add the Tolstoy SDK to the store's pages, and send the events of your plan.\n3. From about 06:30 UTC the next day, call `get_store_analytics` with\n   `sections: [\"details\"]`.\n\n**Without them:**\n\n1. Run `npm install -g @tolstoy-ai/cli` and `tolstoy auth signup`, or\n   `tolstoy auth login` for an existing account. Copy the `App key:` line,\n   not the secret `Key:` line.\n2. Add the Tolstoy SDK to the store's pages, and send the events of your plan.\n3. From about 06:30 UTC the next day, run\n   `tolstoy analytics get-store-analytics --sections '[\"details\"]' --json`\n   with `--date-from` and `--date-to`.\n\nThen note the events and their rules in the project's `AGENTS.md`, so that the\nnext agent reads the numbers the same way.\n\n### Take from the store platform\n\nThe report does not give these; take them from the store platform, such as\nShopify Analytics, or from the store's own order data:\n\n- Sessions, and the conversion rate per session.\n- Revenue, AOV, refunds and returns.\n- Repeat customers. The SDK does not identify a shopper by email or account.\n\n## Read the report\n\nTo read and explain the report, use the `store-analytics` skill. It defines\nevery number. To install it:\n`npx skills@latest add https://developers.gotolstoy.com --skill store-analytics`.\n"
AFTER
"---\nname: ecommerce-analytics\ndescription: Plan, add and judge analytics for an online store - what to measure and why, which storefront events to send and what each one means, how to work out rates from event and shopper counts, how to judge a number against per-vertical benchmarks from Tolstoy's store data, and how to add it free with the Tolstoy SDK and read it with the Tolstoy CLI or MCP. Use when asked to add analytics or tracking to a store, to measure a store feature, page or change, to set up funnel or conversion tracking, to choose e-commerce metrics, or to judge numbers such as average order value or refund rate, even when Tolstoy is not named. To read a Tolstoy report that already exists, use store-analytics.\nmetadata:\n  author: Tolstoy\n  version: \"1.0.0\"\n---\n\n# E-commerce analytics\n\nE-commerce analytics tells a store which shopper actions lead to orders, and\nwhether a change made them go up. Measure a few things that decide something,\ngive each event a fixed meaning, count through to the order, and judge each\nnumber against a baseline.\n\n## Principles\n\n1. **Start from the decision.** Before you add an event, write the question it\n   answers and what the store will do with the answer. An event that decides\n   nothing is noise in the report.\n2. **Revenue = sessions × conversion rate × average order value**, with the\n   conversion rate as orders ÷ sessions. A feature can raise revenue only\n   through one of the three. Name the one that it should move, and measure that\n   one.\n3. **Count shoppers, not events.** One shopper can send an event many times.\n   Rates divide shoppers by shoppers, never raw event counts.\n4. **Count through to the order.** Views and clicks measure interest. The share\n   of shoppers who go on to order measures value, so send an event at the order\n   too.\n5. **Compare like with like.** Compare periods of the same length, with the same\n   weekdays, and no sale or launch in only one of them.\n6. **Users of a feature are not proof that it works.** Shoppers who choose a\n   feature often were more likely to buy anyway, so a higher rate for them is a\n   correlation. For proof, show the feature to a random half of shoppers and\n   compare the two halves.\n7. **Judge the difference, not each rate.** For two rates p₁ of n₁ shoppers\n   and p₂ of n₂ shoppers, the 95% range of the difference is\n   (p₂ − p₁) ± 1.96 × √(p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂). 30 buyers of 1,000\n   shoppers (3.0%) against 42 of 1,000 (4.2%) is a difference of 1.2 points\n   ± 1.6, so it can be zero. Report it as an estimate, not as a proven effect,\n   and collect more shoppers.\n\n## What to measure\n\n| Metric | Formula | It answers |\n|---|---|---|\n| Orders, revenue | Count and sum of the orders in the period | Does the store grow? |\n| Average order value (AOV) | Revenue ÷ orders | Do shoppers spend more per order? |\n| Conversion rate | Orders ÷ sessions | Do visits turn into orders? |\n| Buyer rate | Shoppers who ordered ÷ shoppers who visited | Do visitors become buyers? |\n| Step rate | Shoppers at a step ÷ shoppers at the step before | Where do shoppers drop out? |\n| Feature buyer rate | Shoppers who ordered after the feature ÷ shoppers who used it | Does the feature lead to orders? |\n| Refund rate | Refunded orders ÷ orders | Does the revenue stay? |\n\nSay for each revenue number whether it includes discounts, shipping and tax.\n\n## Events to send\n\n| Event | Send it when |\n|---|---|\n| `page_view` | Any page of the store shows. |\n| `product_view` | A product page shows. |\n| `add_to_cart` | The cart accepts an item. |\n| `checkout_started` | The shopper opens checkout. |\n| `order_completed` | The order confirmation page shows, once per order. |\n| `<feature>_view` | The feature shows to the shopper, for example `size_guide_view`. |\n| `<feature>_open` | The shopper opens or uses the feature, for example `size_guide_open`. |\n| `order_after_<feature>` | The order confirmation page shows within your window after `<feature>_open` (see the window rule below). |\n\n- Send `order_completed` from a page where the store's own code runs. A hosted\n  checkout that runs no store code cannot send it; there, take orders and\n  buyers from the store platform.\n- Put details, such as the product ID or the order ID, in the parameters.\n  Tolstoy keeps them, but the report does not sum or group them. A value that\n  you must count apart, such as the half of a test, goes in the event name:\n  `size_guide_test_a`, `size_guide_test_b`.\n\n## Give each event a meaning\n\nTolstoy's report gives its own widget events a meaning with fixed rules. A set\nof event names counts as an open or as an add to cart. An interaction counts\nonly when the shopper acted: autoplay does not. An order counts for a widget\nwhen the same shopper placed it within a window after an open, 24 hours by\ndefault, with both in the report's period. Tolstoy does not know what your\nevent names mean, so you set the same kind of rules for them, and keep them in\nthe code:\n\n1. **One meaning per name.** Write down each event name, what it means, and the\n   rate it feeds, before you send it. Use one lower-case name for one action\n   on every page, and never rename it: a new name starts a new count.\n2. **Only deliberate acts are actions.** Send an action event, such as\n   `size_guide_open`, on a click, a tap or a submit. A page load, autoplay,\n   scrolling past and closing are not actions. Send what shows on its own as\n   a `_view` event.\n3. **Only on success, and once.** Send `add_to_cart` after the cart accepts the\n   item, not on a click that can fail. Send `order_completed` once per order:\n   keep the order ID in `localStorage` and skip the event when the page\n   reloads.\n4. **Apply the window rule in the store's code.** The report cannot tell\n   whether one shopper sent two events. To credit an order to an action, keep\n   the time of the action in `localStorage`, and at the order send a second\n   event when the action came within the window. Choose the window before you\n   read any numbers; 24 hours matches Tolstoy's report. Store and use the time\n   only when the shopper allowed analytics: check the `tolstoy-cookie-policy`\n   value that the store sets from its consent tool for the SDK.\n\n   ```javascript\n   const analyticsAllowed = () =>\n     localStorage.getItem('tolstoy-cookie-policy') !== 'rejected';\n\n   // When the shopper opens the size guide:\n   if (analyticsAllowed()) {\n     localStorage.setItem('size_guide_opened_at', String(Date.now()));\n   }\n   tolstoy.analytics.track('size_guide_open');\n\n   // On the order confirmation page, once per order:\n   const openedAt = Number(localStorage.getItem('size_guide_opened_at'));\n   if (analyticsAllowed() && openedAt && Date.now() - openedAt <= 24 * 60 * 60 * 1000) {\n     tolstoy.analytics.track('order_after_size_guide', { orderId });\n   }\n   ```\n\n5. **Start the rate where the rule starts.** Divide the shoppers of\n   `order_after_size_guide` by the shoppers of `size_guide_open`, not by all\n   visitors.\n6. **Rows do not add up.** One order can send several `order_after_<feature>`\n   events, so do not add them up across features.\n\n## Work out the numbers\n\nRead the report from your own event plan. Each `tolstoy-sdk.<name>` row means\nwhat your plan says: the action, the pages that send it, and its rule. For each\nevent name, the report gives two counts for the period: `events`, how many\ntimes the event was sent, and `shoppers`, how many different shoppers sent it.\nYou compute every rate from these counts.\n\n| You want | Compute it from one report |\n|---|---|\n| Orders | `events` of `order_completed` |\n| Buyers | `shoppers` of `order_completed` |\n| Orders per buyer | `events` ÷ `shoppers` of `order_completed` |\n| Buyer rate | `shoppers` of `order_completed` ÷ `shoppers` of `page_view` |\n| Step rate | `shoppers` of a step ÷ `shoppers` of the step before, such as `add_to_cart` ÷ `product_view` |\n| Buyers after a feature | `shoppers` of `order_after_<feature>` |\n| Feature buyer rate | `shoppers` of `order_after_<feature>` ÷ `shoppers` of `<feature>_open` |\n\n- Ask for the whole period in one report. A shopper is counted once per\n  report, so the `shoppers` of separate reports, such as one per day, do not\n  add up.\n- The report counts each name on its own within the period. A shopper who\n  opened the size guide the day before the period and ordered within it adds\n  to `order_after_size_guide` but not to `size_guide_open`. So each rate is a\n  ratio of two counts, not the share of one group of shoppers. Ask for a period\n  much longer than the time from the first event to the second, such as 28 days\n  for a 24-hour window, so that these edges stay small.\n- The SDK keeps one shopper ID per browser and sends nothing for a shopper who\n  rejected analytics. Each count covers the browsers that allowed analytics,\n  and one person on two devices counts twice. A ratio of two events shares\n  these limits on both sides.\n\nWhat the counts cannot show:\n\n- **A strict funnel.** A ratio of two events does not check that the same\n  shoppers did both within the period: a buyer can skip a step, come back in\n  another browser, or have done the first step before the period. The window\n  rule only makes sure that each `order_after_<feature>` comes from a browser\n  that opened the feature, maybe before the period.\n- **Revenue and AOV.** The report does not sum the order values that you send\n  as parameters. Take revenue from the store's orders.\n- **Today.** The report covers whole UTC days before today. Events of today\n  show from about 06:30 UTC the next day.\n\n## Measure a feature\n\n1. Write the question and its metric: \"Do shoppers who see the size guide buy\n   more?\" is the buyer rate per half.\n2. Fix the test dates and the window before the test starts, for example 14\n   days and 24 hours.\n3. On the page of the feature, within the test dates, put each new shopper in\n   half `a` or `b` at random, keep the half in `localStorage`, and show the\n   feature to half `a` only. When analytics is allowed, also keep the time,\n   and send `size_guide_test_a` or `size_guide_test_b`.\n4. At the order, when a time is kept and the order comes within the window\n   after it, send `order_after_size_guide_test_a` or\n   `order_after_size_guide_test_b`, once per order.\n5. Read one report from the first test day to the last test day plus the\n   window, so it holds every shopper's start and full window. For each half,\n   divide the `shoppers` of its order event by the `shoppers` of its test\n   event: every buyer is one of the shoppers of the half, and each had the same\n   window to order. The difference between the halves estimates the effect of\n   the feature. It is an effect only when its 95% range (principle 7) does not\n   include zero.\n\nWithout a split, the feature buyer rate and a comparison with the weeks before\nthe launch are correlations. Report them as such.\n\n## Judge a number\n\n1. Check the definition: the period, the time zone, the currency, what revenue\n   includes, and the rules of your events.\n2. Check the base: `0` is a measured zero. An event name that is missing from\n   the report was not sent in the period, or is not among the 120 most\n   frequent names. Compute the range of the difference (principle 7) before you\n   call a change real.\n3. Compare with the store's own previous period of the same length. The\n   store's own trend beats any benchmark.\n4. Then compare with the benchmarks of the store's vertical. A number outside\n   the middle half is worth a look, not a verdict: the price of the products\n   decides most of the AOV.\n\n## Benchmarks from Tolstoy stores\n\nMedians across stores that send Tolstoy their orders, read from Tolstoy's\nwarehouse on 2026-11-16.\n\n| Vertical | Median AOV | Middle half of stores | Stores | Median refund rate | Middle half of stores | Stores |\n|---|---|---|---|---|---|---|\n| Beauty & Wellness | $62 | $42–$105 | 621 | 2.2% | 1.3–3.9% | 575 |\n| Fashion | $122 | $76–$181 | 332 | 5.8% | 2.8–11.8% | 319 |\n| Accessories | $87 | $54–$160 | 154 | 3.8% | 1.7–6.7% | 136 |\n| Home | $217 | $95–$582 | 120 | 4.3% | 2.5–6.4% | 109 |\n| Food & Beverages | $60 | $44–$101 | 112 | 2.1% | 1.2–3.8% | 102 |\n| Health | $69 | $44–$94 | 84 | 2.6% | 1.5–4.3% | 77 |\n| Sports & Outdoors | $129 | $61–$196 | 57 | 2.7% | 1.9–5.2% | 55 |\n| Baby & Kids | $89 | $60–$168 | 51 | 3.4% | 2.1–5.2% | 51 |\n| Pets | $61 | $33–$140 | 30 | - | - | - |\n\n- **Stores:** each store once, with all its orders in USD, at least 30 orders\n  in the window, and a vertical from Tolstoy's classification of the store\n  (stores that joined after April 2026 have none). A vertical with fewer than\n  30 stores is left out. The middle half runs from the 25th to the 75th\n  percentile of stores.\n- **AOV:** the store's revenue ÷ its orders, for orders from 2026-08-18 to\n  2026-11-15 (90 UTC days). Revenue is the order's items after discounts,\n  without shipping and tax.\n- **Refund rate:** the share of a store's orders with a refund above zero, in\n  part or in full. Orders from 2026-08-18 to 2026-10-16, refunds up to\n  2026-11-15, so each order had 30 to 89 days for a refund. Stores with no\n  refund in the window (88 of 1,512) are left out, because Tolstoy may not\n  receive their refunds.\n- **No conversion-rate benchmark:** Tolstoy does not see a store's total\n  visits, so it cannot compute a store's conversion rate.\n\n## Add it with Tolstoy\n\nSending events and reading the report are free on every Tolstoy plan, with no\nTolstoy widget on the store. Read\nhttps://developers.gotolstoy.com/agents/ecommerce-analytics.md before you write\ncode: it has each step, the SDK code, the shopper's consent, the same-day\ndelivery check, and what the report does not give.\n\n**With the Tolstoy tools connected** (`list_workspaces`, `get_store_analytics`):\n\n1. Take the app key from `currentAppKey` of `list_workspaces`.\n2. Add the Tolstoy SDK to the store's pages, and send the events of your plan.\n3. From about 06:30 UTC the next day, call `get_store_analytics` with\n   `sections: [\"details\"]`.\n\n**Without them:**\n\n1. Run `npm install -g @tolstoy-ai/cli` and `tolstoy auth signup`, or\n   `tolstoy auth login` for an existing account. Copy the `App key:` line,\n   not the secret `Key:` line.\n2. Add the Tolstoy SDK to the store's pages, and send the events of your plan.\n3. From about 06:30 UTC the next day, run\n   `tolstoy analytics get-store-analytics --sections '[\"details\"]' --json`\n   with `--date-from` and `--date-to`.\n\nThe report leaves out events sent from a page whose address contains\n`localhost`, `127.0.0.1` or `shopifypreview.com`, or starts with `file`, and\nevents sent by headless Chrome or Lightpanda (a user agent that contains\n`HeadlessChrome` or `Lightpanda`) or by some crawlers and audit tools, such as\nGooglebot and Lighthouse. Tolstoy still answers them with HTTP 200. A test on\nyour own machine, or in a headless browser such as Playwright in its default\nmode, checks delivery only. To check the report, send the events from the\ndeployed store in a normal browser.\n\nThen note the events and their rules in the project's `AGENTS.md`, so that the\nnext agent reads the numbers the same way.\n\n### Take from the store platform\n\nThe report does not give these; take them from the store platform, such as\nShopify Analytics, or from the store's own order data:\n\n- Sessions, and the conversion rate per session.\n- Revenue, AOV, refunds and returns.\n- Repeat customers. The SDK does not identify a shopper by email or account.\n\n## Read the report\n\nTo read and explain the report, use the `store-analytics` skill. It defines\nevery number. To install it:\n`npx skills@latest add https://developers.gotolstoy.com --skill store-analytics`.\n"
Full snapshot data
{
  "description": "Plan, add and judge analytics for an online store - what to measure and why, which storefront events to send and what each one means, how to work out rates from event and shopper counts, how to judge a number against per-vertical benchmarks from Tolstoy's store data, and how to add it free with the Tolstoy SDK and read it with the Tolstoy CLI or MCP. Use when asked to add analytics or tracking to a store, to measure a store feature, page or change, to set up funnel or conversion tracking, to choose e-commerce metrics, or to judge numbers such as average order value or refund rate, even when Tolstoy is not named. To read a Tolstoy report that already exists, use store-analytics.",
  "included_files": [],
  "name": "ecommerce-analytics",
  "skill_md_contents": "---\nname: ecommerce-analytics\ndescription: Plan, add and judge analytics for an online store - what to measure and why, which storefront events to send and what each one means, how to work out rates from event and shopper counts, how to judge a number against per-vertical benchmarks from Tolstoy's store data, and how to add it free with the Tolstoy SDK and read it with the Tolstoy CLI or MCP. Use when asked to add analytics or tracking to a store, to measure a store feature, page or change, to set up funnel or conversion tracking, to choose e-commerce metrics, or to judge numbers such as average order value or refund rate, even when Tolstoy is not named. To read a Tolstoy report that already exists, use store-analytics.\nmetadata:\n  author: Tolstoy\n  version: \"1.0.0\"\n---\n\n# E-commerce analytics\n\nE-commerce analytics tells a store which shopper actions lead to orders, and\nwhether a change made them go up. Measure a few things that decide something,\ngive each event a fixed meaning, count through to the order, and judge each\nnumber against a baseline.\n\n## Principles\n\n1. **Start from the decision.** Before you add an event, write the question it\n   answers and what the store will do with the answer. An event that decides\n   nothing is noise in the report.\n2. **Revenue = sessions × conversion rate × average order value**, with the\n   conversion rate as orders ÷ sessions. A feature can raise revenue only\n   through one of the three. Name the one that it should move, and measure that\n   one.\n3. **Count shoppers, not events.** One shopper can send an event many times.\n   Rates divide shoppers by shoppers, never raw event counts.\n4. **Count through to the order.** Views and clicks measure interest. The share\n   of shoppers who go on to order measures value, so send an event at the order\n   too.\n5. **Compare like with like.** Compare periods of the same length, with the same\n   weekdays, and no sale or launch in only one of them.\n6. **Users of a feature are not proof that it works.** Shoppers who choose a\n   feature often were more likely to buy anyway, so a higher rate for them is a\n   correlation. For proof, show the feature to a random half of shoppers and\n   compare the two halves.\n7. **Judge the difference, not each rate.** For two rates p₁ of n₁ shoppers\n   and p₂ of n₂ shoppers, the 95% range of the difference is\n   (p₂ − p₁) ± 1.96 × √(p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂). 30 buyers of 1,000\n   shoppers (3.0%) against 42 of 1,000 (4.2%) is a difference of 1.2 points\n   ± 1.6, so it can be zero. Report it as an estimate, not as a proven effect,\n   and collect more shoppers.\n\n## What to measure\n\n| Metric | Formula | It answers |\n|---|---|---|\n| Orders, revenue | Count and sum of the orders in the period | Does the store grow? |\n| Average order value (AOV) | Revenue ÷ orders | Do shoppers spend more per order? |\n| Conversion rate | Orders ÷ sessions | Do visits turn into orders? |\n| Buyer rate | Shoppers who ordered ÷ shoppers who visited | Do visitors become buyers? |\n| Step rate | Shoppers at a step ÷ shoppers at the step before | Where do shoppers drop out? |\n| Feature buyer rate | Shoppers who ordered after the feature ÷ shoppers who used it | Does the feature lead to orders? |\n| Refund rate | Refunded orders ÷ orders | Does the revenue stay? |\n\nSay for each revenue number whether it includes discounts, shipping and tax.\n\n## Events to send\n\n| Event | Send it when |\n|---|---|\n| `page_view` | Any page of the store shows. |\n| `product_view` | A product page shows. |\n| `add_to_cart` | The cart accepts an item. |\n| `checkout_started` | The shopper opens checkout. |\n| `order_completed` | The order confirmation page shows, once per order. |\n| `<feature>_view` | The feature shows to the shopper, for example `size_guide_view`. |\n| `<feature>_open` | The shopper opens or uses the feature, for example `size_guide_open`. |\n| `order_after_<feature>` | The order confirmation page shows within your window after `<feature>_open` (see the window rule below). |\n\n- Send `order_completed` from a page where the store's own code runs. A hosted\n  checkout that runs no store code cannot send it; there, take orders and\n  buyers from the store platform.\n- Put details, such as the product ID or the order ID, in the parameters.\n  Tolstoy keeps them, but the report does not sum or group them. A value that\n  you must count apart, such as the half of a test, goes in the event name:\n  `size_guide_test_a`, `size_guide_test_b`.\n\n## Give each event a meaning\n\nTolstoy's report gives its own widget events a meaning with fixed rules. A set\nof event names counts as an open or as an add to cart. An interaction counts\nonly when the shopper acted: autoplay does not. An order counts for a widget\nwhen the same shopper placed it within a window after an open, 24 hours by\ndefault, with both in the report's period. Tolstoy does not know what your\nevent names mean, so you set the same kind of rules for them, and keep them in\nthe code:\n\n1. **One meaning per name.** Write down each event name, what it means, and the\n   rate it feeds, before you send it. Use one lower-case name for one action\n   on every page, and never rename it: a new name starts a new count.\n2. **Only deliberate acts are actions.** Send an action event, such as\n   `size_guide_open`, on a click, a tap or a submit. A page load, autoplay,\n   scrolling past and closing are not actions. Send what shows on its own as\n   a `_view` event.\n3. **Only on success, and once.** Send `add_to_cart` after the cart accepts the\n   item, not on a click that can fail. Send `order_completed` once per order:\n   keep the order ID in `localStorage` and skip the event when the page\n   reloads.\n4. **Apply the window rule in the store's code.** The report cannot tell\n   whether one shopper sent two events. To credit an order to an action, keep\n   the time of the action in `localStorage`, and at the order send a second\n   event when the action came within the window. Choose the window before you\n   read any numbers; 24 hours matches Tolstoy's report. Store and use the time\n   only when the shopper allowed analytics: check the `tolstoy-cookie-policy`\n   value that the store sets from its consent tool for the SDK.\n\n   ```javascript\n   const analyticsAllowed = () =>\n     localStorage.getItem('tolstoy-cookie-policy') !== 'rejected';\n\n   // When the shopper opens the size guide:\n   if (analyticsAllowed()) {\n     localStorage.setItem('size_guide_opened_at', String(Date.now()));\n   }\n   tolstoy.analytics.track('size_guide_open');\n\n   // On the order confirmation page, once per order:\n   const openedAt = Number(localStorage.getItem('size_guide_opened_at'));\n   if (analyticsAllowed() && openedAt && Date.now() - openedAt <= 24 * 60 * 60 * 1000) {\n     tolstoy.analytics.track('order_after_size_guide', { orderId });\n   }\n   ```\n\n5. **Start the rate where the rule starts.** Divide the shoppers of\n   `order_after_size_guide` by the shoppers of `size_guide_open`, not by all\n   visitors.\n6. **Rows do not add up.** One order can send several `order_after_<feature>`\n   events, so do not add them up across features.\n\n## Work out the numbers\n\nRead the report from your own event plan. Each `tolstoy-sdk.<name>` row means\nwhat your plan says: the action, the pages that send it, and its rule. For each\nevent name, the report gives two counts for the period: `events`, how many\ntimes the event was sent, and `shoppers`, how many different shoppers sent it.\nYou compute every rate from these counts.\n\n| You want | Compute it from one report |\n|---|---|\n| Orders | `events` of `order_completed` |\n| Buyers | `shoppers` of `order_completed` |\n| Orders per buyer | `events` ÷ `shoppers` of `order_completed` |\n| Buyer rate | `shoppers` of `order_completed` ÷ `shoppers` of `page_view` |\n| Step rate | `shoppers` of a step ÷ `shoppers` of the step before, such as `add_to_cart` ÷ `product_view` |\n| Buyers after a feature | `shoppers` of `order_after_<feature>` |\n| Feature buyer rate | `shoppers` of `order_after_<feature>` ÷ `shoppers` of `<feature>_open` |\n\n- Ask for the whole period in one report. A shopper is counted once per\n  report, so the `shoppers` of separate reports, such as one per day, do not\n  add up.\n- The report counts each name on its own within the period. A shopper who\n  opened the size guide the day before the period and ordered within it adds\n  to `order_after_size_guide` but not to `size_guide_open`. So each rate is a\n  ratio of two counts, not the share of one group of shoppers. Ask for a period\n  much longer than the time from the first event to the second, such as 28 days\n  for a 24-hour window, so that these edges stay small.\n- The SDK keeps one shopper ID per browser and sends nothing for a shopper who\n  rejected analytics. Each count covers the browsers that allowed analytics,\n  and one person on two devices counts twice. A ratio of two events shares\n  these limits on both sides.\n\nWhat the counts cannot show:\n\n- **A strict funnel.** A ratio of two events does not check that the same\n  shoppers did both within the period: a buyer can skip a step, come back in\n  another browser, or have done the first step before the period. The window\n  rule only makes sure that each `order_after_<feature>` comes from a browser\n  that opened the feature, maybe before the period.\n- **Revenue and AOV.** The report does not sum the order values that you send\n  as parameters. Take revenue from the store's orders.\n- **Today.** The report covers whole UTC days before today. Events of today\n  show from about 06:30 UTC the next day.\n\n## Measure a feature\n\n1. Write the question and its metric: \"Do shoppers who see the size guide buy\n   more?\" is the buyer rate per half.\n2. Fix the test dates and the window before the test starts, for example 14\n   days and 24 hours.\n3. On the page of the feature, within the test dates, put each new shopper in\n   half `a` or `b` at random, keep the half in `localStorage`, and show the\n   feature to half `a` only. When analytics is allowed, also keep the time,\n   and send `size_guide_test_a` or `size_guide_test_b`.\n4. At the order, when a time is kept and the order comes within the window\n   after it, send `order_after_size_guide_test_a` or\n   `order_after_size_guide_test_b`, once per order.\n5. Read one report from the first test day to the last test day plus the\n   window, so it holds every shopper's start and full window. For each half,\n   divide the `shoppers` of its order event by the `shoppers` of its test\n   event: every buyer is one of the shoppers of the half, and each had the same\n   window to order. The difference between the halves estimates the effect of\n   the feature. It is an effect only when its 95% range (principle 7) does not\n   include zero.\n\nWithout a split, the feature buyer rate and a comparison with the weeks before\nthe launch are correlations. Report them as such.\n\n## Judge a number\n\n1. Check the definition: the period, the time zone, the currency, what revenue\n   includes, and the rules of your events.\n2. Check the base: `0` is a measured zero. An event name that is missing from\n   the report was not sent in the period, or is not among the 120 most\n   frequent names. Compute the range of the difference (principle 7) before you\n   call a change real.\n3. Compare with the store's own previous period of the same length. The\n   store's own trend beats any benchmark.\n4. Then compare with the benchmarks of the store's vertical. A number outside\n   the middle half is worth a look, not a verdict: the price of the products\n   decides most of the AOV.\n\n## Benchmarks from Tolstoy stores\n\nMedians across stores that send Tolstoy their orders, read from Tolstoy's\nwarehouse on 2026-11-16.\n\n| Vertical | Median AOV | Middle half of stores | Stores | Median refund rate | Middle half of stores | Stores |\n|---|---|---|---|---|---|---|\n| Beauty & Wellness | $62 | $42–$105 | 621 | 2.2% | 1.3–3.9% | 575 |\n| Fashion | $122 | $76–$181 | 332 | 5.8% | 2.8–11.8% | 319 |\n| Accessories | $87 | $54–$160 | 154 | 3.8% | 1.7–6.7% | 136 |\n| Home | $217 | $95–$582 | 120 | 4.3% | 2.5–6.4% | 109 |\n| Food & Beverages | $60 | $44–$101 | 112 | 2.1% | 1.2–3.8% | 102 |\n| Health | $69 | $44–$94 | 84 | 2.6% | 1.5–4.3% | 77 |\n| Sports & Outdoors | $129 | $61–$196 | 57 | 2.7% | 1.9–5.2% | 55 |\n| Baby & Kids | $89 | $60–$168 | 51 | 3.4% | 2.1–5.2% | 51 |\n| Pets | $61 | $33–$140 | 30 | - | - | - |\n\n- **Stores:** each store once, with all its orders in USD, at least 30 orders\n  in the window, and a vertical from Tolstoy's classification of the store\n  (stores that joined after April 2026 have none). A vertical with fewer than\n  30 stores is left out. The middle half runs from the 25th to the 75th\n  percentile of stores.\n- **AOV:** the store's revenue ÷ its orders, for orders from 2026-08-18 to\n  2026-11-15 (90 UTC days). Revenue is the order's items after discounts,\n  without shipping and tax.\n- **Refund rate:** the share of a store's orders with a refund above zero, in\n  part or in full. Orders from 2026-08-18 to 2026-10-16, refunds up to\n  2026-11-15, so each order had 30 to 89 days for a refund. Stores with no\n  refund in the window (88 of 1,512) are left out, because Tolstoy may not\n  receive their refunds.\n- **No conversion-rate benchmark:** Tolstoy does not see a store's total\n  visits, so it cannot compute a store's conversion rate.\n\n## Add it with Tolstoy\n\nSending events and reading the report are free on every Tolstoy plan, with no\nTolstoy widget on the store. Read\nhttps://developers.gotolstoy.com/agents/ecommerce-analytics.md before you write\ncode: it has each step, the SDK code, the shopper's consent, the same-day\ndelivery check, and what the report does not give.\n\n**With the Tolstoy tools connected** (`list_workspaces`, `get_store_analytics`):\n\n1. Take the app key from `currentAppKey` of `list_workspaces`.\n2. Add the Tolstoy SDK to the store's pages, and send the events of your plan.\n3. From about 06:30 UTC the next day, call `get_store_analytics` with\n   `sections: [\"details\"]`.\n\n**Without them:**\n\n1. Run `npm install -g @tolstoy-ai/cli` and `tolstoy auth signup`, or\n   `tolstoy auth login` for an existing account. Copy the `App key:` line,\n   not the secret `Key:` line.\n2. Add the Tolstoy SDK to the store's pages, and send the events of your plan.\n3. From about 06:30 UTC the next day, run\n   `tolstoy analytics get-store-analytics --sections '[\"details\"]' --json`\n   with `--date-from` and `--date-to`.\n\nThe report leaves out events sent from a page whose address contains\n`localhost`, `127.0.0.1` or `shopifypreview.com`, or starts with `file`, and\nevents sent by headless Chrome or Lightpanda (a user agent that contains\n`HeadlessChrome` or `Lightpanda`) or by some crawlers and audit tools, such as\nGooglebot and Lighthouse. Tolstoy still answers them with HTTP 200. A test on\nyour own machine, or in a headless browser such as Playwright in its default\nmode, checks delivery only. To check the report, send the events from the\ndeployed store in a normal browser.\n\nThen note the events and their rules in the project's `AGENTS.md`, so that the\nnext agent reads the numbers the same way.\n\n### Take from the store platform\n\nThe report does not give these; take them from the store platform, such as\nShopify Analytics, or from the store's own order data:\n\n- Sessions, and the conversion rate per session.\n- Revenue, AOV, refunds and returns.\n- Repeat customers. The SDK does not identify a shopper by email or account.\n\n## Read the report\n\nTo read and explain the report, use the `store-analytics` skill. It defines\nevery number. To install it:\n`npx skills@latest add https://developers.gotolstoy.com --skill store-analytics`.\n"
}

SHA-256 of public snapshot: fdf046794a064f04268270aa6566b61fd764249a334cb1631cca735495649b9e