Update to Better Stack
Snapshot Nov 13, 2026 · 12:08 UTC · version 1.3.14
Collection source: downloaded plugin package.
Instructions updated for chart-alerts
Skill instructions — Now says: (disk > 90%, error rate > 5%) | `value`, `operator` | | `relative` | Change in % vs. the past, when the user...
Observed in instructions or declared skills. Runtime behavior has not been tested.
Up to 3 plugins free · no card required.
All fields and original values.
Skill instructions
Nov 11, 2026 · 06:08 UTC
(CPU > 90%, errors > 100) | `value`, `operator` | | `relative` | Value changes by percentage vs. the past (traffic drops 50%) | `value`, `operator`, `query_period` ...
Nov 13, 2026 · 12:08 UTC
(disk > 90%, error rate > 5%) | `value`, `operator` | | `relative` | Change in % vs. the past, when the user asks for one | `value`, `operator`, `query_period` ...
Compare saved observations
Download comparison JSONSKILL.md text changes
− removed · + added
--- before +++ after @@ -13,9 +13,25 @@ | Type | Use Case | Key Parameters | | -------------- | ------------------------------------------------------------ | ---------------------------------------------------------- | -| `threshold` | Known fixed limit exceeded (CPU > 90%, errors > 100) | `value`, `operator` | -| `relative` | Value changes by percentage vs. the past (traffic drops 50%) | `value`, `operator`, `query_period` | -| `anomaly_rrcf` | Unpredictable patterns, no known threshold (ML-based) | `shown_interval`, `anomaly_sensitivity`, `anomaly_trigger` | +| `threshold` | Known fixed limit exceeded (disk > 90%, error rate > 5%) | `value`, `operator` | +| `relative` | Change in % vs. the past, when the user asks for one | `value`, `operator`, `query_period` | +| `anomaly_rrcf` | Unusual patterns (ML-based), with a floor by default (below) | `shown_interval`, `anomaly_sensitivity`, `anomaly_trigger` | + +## What to Alert On + +Unless the user asks otherwise, every alert fires only on something both bad and unusual: + +- **Outcomes, not activity**: orders completed, jobs finished, age of the oldest queued job. A stuck + worker that retries every 2 s keeps emitting spans and requests while nothing gets done. +- **Saturation** (memory, disk, connection pools): `threshold`, since it is bad at the same level every day. +- **Customer impact** (error rate, latency, failed checkouts, orders dropping): `threshold`; for a + drop, `lower_than` a value under the quietest normal hour. An anomaly or relative alert learns a + sustained change as the new normal and resolves within about an hour while the outage goes on. +- **Anything else unusual**: `anomaly_rrcf` with a floor. +- **Slices**: group error and latency charts by endpoint or error type. Each series is checked on its + own, so a failure in one slice crosses the bar instead of hiding in the whole-traffic average. +- **Low-traffic services** (a few requests per second or fewer): a ratio jumps on a handful of + requests; alert on error counts instead. ## Operators @@ -47,14 +63,16 @@ ### Value (Threshold/Relative) -**Critical**: The `value` must match the data format returned by the chart query. +**Critical**: The `value` is compared with the raw numbers the chart query returns, not with what the chart displays. -| Chart Data Format | Example Query | Alert Value for "85%" | -| ------------------ | --------------------------------------------------------------- | --------------------- | -| Percentage (0-100) | `avg(cpu_percent)` | `85` | -| Ratio (0-1) | `sumIf(logs_count, label('level') = 'error') / sum(logs_count)` | `0.85` | +| Chart | Example Query | Alert Value for "85%" | +| ---------------------------------------------- | --------------------------------------------------------------- | --------------------- | +| `unit: "percent"` | `sumIf(logs_count, label('level') = 'error') / sum(logs_count)` | `0.85` | +| Data already 0-100, without the `percent` unit | `avg(cpu_percent)` | `85` | -**Before creating an alert**, check the chart's actual values using `chart` or look at the dashboard to understand the data range. +A `percent` chart holds 0-1 ratios and multiplies them by 100 for display, so it shows 0.85 as 85% while its alert `value` stays `0.85`. The exception is a `stack_to_100` bar chart: it sets `percent` itself but its query returns raw values, so its thresholds are raw values too. A relative alert's `value` stays a percentage change either way (50 means 50%). + +**Before creating an alert**, check the chart's `unit` and actual values using `chart`. ### Confirmation Period @@ -67,6 +85,15 @@ - Set equal to or higher than confirmation_period - Prevents alert flapping +### Missing Data + +`on_missing_data` is what a threshold or relative alert does when its chart returns no data: `dont_fire` +(default), `treat_as_zero`, `treat_as_previous`, or `start_incident`. Use `start_incident` on throughput and +outcome charts (requests, orders completed): when that data stops, the service is down, and `dont_fire` +would resolve any open incident and stay quiet. On a sparse chart (a few orders an hour), empty buckets +in quiet hours open incidents too: widen `aggregation_interval` there, or keep `dont_fire`. Anomaly +alerts take it from the chart and refuse it. + ### Query Period (Relative Alerts) How far back to look for comparison. Fixed buckets (seconds): `30, 60, 120, 300, 600, 1800, 3600` — **capped at 1 hour**; "vs 1 day/week ago" comparisons are not possible. @@ -81,6 +108,31 @@ Don't guess a significance or a threshold: query the chart's data over the past days first (`chart`, then `query`) and say how often the condition would have been met. +## Extra Conditions: Unusual AND Bad + +`additional_conditions` ANDs up to 4 more conditions onto the alert's own: it fires only when every +condition breaches in the same time bucket, on any of the series it covers, not necessarily the same +one. On a grouped chart, an unscoped floor is met by whichever series crosses it. Each is `{alert_type, operator, value, series_names, +series_names_except}`, `threshold` or `relative` only (anomaly detection can be the alert's own type, +not an extra condition), and each may scope its own series. + +**Give every anomaly alert a floor by default.** On its own an anomaly alert fires on anything +unusual, including harmless shifts: after a node replacement, CPU moving to 21% or a p95 of 76 ms is +unusual but fine. A threshold condition such as "and CPU > 80" or "and p95 > 500 ms" keeps it to what +is both unusual and bad. Pick the floor from the chart's real values (`chart`), the way you would a +threshold alert's `value`. + +On `edit_chart_alert` the list replaces the alert's conditions as a whole: omit it to keep them, +pass `[]` to remove them. `chart_alert` lists them as **Additional Conditions**. + +Check a floor the way you would a threshold: query the chart's data over the past days (`chart`, then `query`) and count how often it crosses the floor. The alert fires at most where the metric is both unusual and past the floor. + +## Metadata + +`metadata` tags the alert, e.g. `{severity: "critical", owner: ["payments"]}`: each value a string or +a list of strings. The incidents the alert opens carry them. On `edit_chart_alert` it replaces the +alert's metadata as a whole; `{}` removes it. + ## Series Names Monitor specific series or all: @@ -95,7 +147,7 @@ | -------------------------- | --------------------------------------------------------- | | `current_team` | Default on-call for the team | | `gid://uptime/Policy/<id>` | A specific escalation policy (from `escalation_policies`) | -| `gid://uptime/Team/<id>` | A specific team's default on-call | +| `gid://uptime/Team/<id>` | A specific team's default on-call | To match a Datadog monitor's notification routing, call `escalation_policies` and pass the closest policy's gid. @@ -108,57 +160,64 @@ | `sms` | `false` | Urgent issues | | `call` | `false` | Critical outages | +Unless the user asks otherwise, call, SMS and a `critical` severity are for customer impact only. An +alert that the chart's history shows firing at the same time every day (a batch job, a backup, a deploy) is a +known event: raise its threshold or floor above it. + ## Process -Find or create the chart (`dashboard` / `add_chart_to_dashboard`) → verify it supports alerts and check its value range (`chart`, then `query` over a few days) → choose the alert type → `create_chart_alert`. +Find or create the chart (`dashboard` / `add_chart_to_dashboard`); an error or latency chart is grouped by endpoint or error type, and on a low-traffic service it counts errors → verify it supports alerts and check its value range (`chart`, then `query` over a few days) → choose the alert type → `create_chart_alert` → tell the user how often each alert's conditions were met over the past week (`query`). ## Examples -### High CPU Alert +### Disk Almost Full Alert ``` chart_id: 123 -name: "High CPU Usage" +name: "Disk almost full" alert_type: "threshold" operator: "higher_than" -value: 0.90 # Use 90 if chart shows 0-100, use 0.90 if chart shows 0-1 +value: 0.90 # On a `percent` chart (0-1 data shown as 90%); 90 only if the query returns 0-100 confirmation_period: 60 ``` -### Traffic Drop Alert +### Orders Stopped Alert ``` chart_id: 123 -name: "Traffic Drop" -alert_type: "relative" -operator: "decreases_by" -value: 50 -query_period: 3600 -confirmation_period: 300 +name: "Orders stopped" +alert_type: "threshold" +operator: "lower_than" +value: 5 # Orders per minute, under the quietest normal hour +confirmation_period: 600 ``` -### Anomaly Detection +### Anomaly Detection with a Floor ``` chart_id: 123 -name: "Unusual Activity" +name: "CPU unusually high" alert_type: "anomaly_rrcf" operator: "higher_than" anomaly_sensitivity: 5 shown_interval: 10800 -anomaly_trigger: "any" +anomaly_trigger: "higher" +additional_conditions: [{ alert_type: "threshold", operator: "higher_than", value: 0.8 }] # 80 only if the query returns 0-100 +metadata: { severity: "warning" } ``` For anomaly alerts the direction comes from `anomaly_trigger` (`any` = both directions), not from `operator`/`value` (threshold parameters). ## Common Mistakes -| Mistake | Correct Approach | -| -------------------------------------------- | ---------------------------------------------- | -| Using relative operators with threshold | `increases_by` only works with `relative` type | +| Mistake | Correct Approach | +| -------------------------------------------- | ---------------------------------------------------------------- | +| Using relative operators with threshold | `increases_by` only works with `relative` type | +| `value: 85` for 85% on a `percent` chart | Its data is 0-1: use `0.85` (the chart and alert show it as 85%) | | Guessing at a significance or threshold | Query the chart's history first, then quote how often it would fire | -| No confirmation period for noisy metrics | Add 60-300s to prevent flapping | -| Alerting on all series when only some matter | Specify `series_names` array | +| No confirmation period for noisy metrics | Add 60-300s to prevent flapping | +| Alerting on all series when only some matter | Specify `series_names` array | +| Anomaly alert with no floor | Add a threshold in `additional_conditions` | ## Managing Alerts @@ -168,5 +227,6 @@ | Get alert details | `chart_alert` | | Create | `create_chart_alert` | | Edit configuration | `edit_chart_alert` | -| Pause/unpause | `toggle_chart_alert_pause` | +| Pause | `pause_chart_alert` | +| Unpause | `unpause_chart_alert` | | Delete | `delete_chart_alert` |
Full technical diff · 1 changed fields
changed /skill_md_contents
"---\nname: chart-alerts\ndescription: Guide for creating and managing Better Stack chart alerts. Load when asked to set up alerting, create alerts on charts, or configure monitoring thresholds.\n---\n\n# Chart Alerts Guide\n\nChart alerts monitor metrics and logs queries and trigger incidents when conditions are met. They are attached to charts on dashboards or explorations. `chart_alert_help` has the server's current reference.\n\n> **Two-tier thresholds:** a chart alert holds a single `value`. To reproduce a Datadog warning + critical monitor, create **two** alerts on the same chart (one per threshold).\n\n## Alert Types\n\n| Type | Use Case | Key Parameters |\n| -------------- | ------------------------------------------------------------ | ---------------------------------------------------------- |\n| `threshold` | Known fixed limit exceeded (CPU > 90%, errors > 100) | `value`, `operator` |\n| `relative` | Value changes by percentage vs. the past (traffic drops 50%) | `value`, `operator`, `query_period` |\n| `anomaly_rrcf` | Unpredictable patterns, no known threshold (ML-based) | `shown_interval`, `anomaly_sensitivity`, `anomaly_trigger` |\n\n## Operators\n\n| Operator | Description | Alert Types |\n| ---------------------- | -------------------------------------- | ------------------- |\n| `higher_than` | Value exceeds threshold | threshold, relative |\n| `lower_than` | Value below threshold | threshold, relative |\n| `higher_than_or_equal` | Value >= threshold | threshold, relative |\n| `lower_than_or_equal` | Value <= threshold | threshold, relative |\n| `equal` | Value equals threshold | threshold, relative |\n| `increases_by` | Value increases by X% | relative only |\n| `decreases_by` | Value decreases by X% | relative only |\n| `changes_by` | Value changes by X% (either direction) | relative only |\n\n## Chart Requirements\n\nOnly certain chart types support alerts:\n\n| Chart Type | Requirements |\n| -------------- | ---------------------------------------------- |\n| `line_chart` | Must be timeseries, query must have `{{time}}` |\n| `bar_chart` | Must be timeseries, query must have `{{time}}` |\n| `number_chart` | Query must have `{{time}}` |\n| `tail_chart` | No additional requirements |\n\n**Prefer line charts on dashboards** for alerts - they provide the best visibility into metric trends and make it easy to correlate alerts with visual data.\n\n## Key Parameters\n\n### Value (Threshold/Relative)\n\n**Critical**: The `value` must match the data format returned by the chart query.\n\n| Chart Data Format | Example Query | Alert Value for \"85%\" |\n| ------------------ | --------------------------------------------------------------- | --------------------- |\n| Percentage (0-100) | `avg(cpu_percent)` | `85` |\n| Ratio (0-1) | `sumIf(logs_count, label('level') = 'error') / sum(logs_count)` | `0.85` |\n\n**Before creating an alert**, check the chart's actual values using `chart` or look at the dashboard to understand the data range.\n\n### Confirmation Period\n\nHow long the condition must persist before triggering an alert. Fixed buckets (seconds): `0, 30, 60, 120, 300, 600, 1800, 3600, 7200, 14400`.\n\n### Recovery Period\n\nHow long the condition must be resolved before the incident is closed. Same fixed buckets as confirmation period, or `null` = no auto-recovery.\n\n- Set equal to or higher than confirmation_period\n- Prevents alert flapping\n\n### Query Period (Relative Alerts)\n\nHow far back to look for comparison. Fixed buckets (seconds): `30, 60, 120, 300, 600, 1800, 3600` — **capped at 1 hour**; \"vs 1 day/week ago\" comparisons are not possible.\n\n### Anomaly Settings\n\n| Parameter | Range | Description |\n| --------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------- |\n| `anomaly_sensitivity` | 1-10 | Counter to its name: HIGHER = fewer alerts, LOWER = more sensitive (default: 5). Shown as **Significance** in the sidebar |\n| `anomaly_trigger` | `any`, `higher`, `lower` | Which anomalies to alert on |\n| `shown_interval` | seconds | Aggregation interval; the training window spans ~10× this (default: 10800 = 3h) |\n\nDon't guess a significance or a threshold: query the chart's data over the past days first (`chart`, then `query`) and say how often the condition would have been met.\n\n## Series Names\n\nMonitor specific series or all:\n\n- Empty array `[]` = monitor all series\n- `[\"production\", \"staging\"]` = only these series\n- Use when you want to alert on specific hosts/services\n\n## Escalation Targets\n\n| Value | Description |\n| -------------------------- | --------------------------------------------------------- |\n| `current_team` | Default on-call for the team |\n| `gid://uptime/Policy/<id>` | A specific escalation policy (from `escalation_policies`) |\n| `gid://uptime/Team/<id>` | A specific team's default on-call |\n\nTo match a Datadog monitor's notification routing, call `escalation_policies` and pass the closest policy's gid.\n\n## Notification Channels\n\n| Channel | Default | Use Case |\n| ------- | ------- | -------------------- |\n| `email` | `true` | Always on |\n| `push` | `false` | Mobile notifications |\n| `sms` | `false` | Urgent issues |\n| `call` | `false` | Critical outages |\n\n## Process\n\nFind or create the chart (`dashboard` / `add_chart_to_dashboard`) → verify it supports alerts and check its value range (`chart`, then `query` over a few days) → choose the alert type → `create_chart_alert`.\n\n## Examples\n\n### High CPU Alert\n\n```\nchart_id: 123\nname: \"High CPU Usage\"\nalert_type: \"threshold\"\noperator: \"higher_than\"\nvalue: 0.90 # Use 90 if chart shows 0-100, use 0.90 if chart shows 0-1\nconfirmation_period: 60\n```\n\n### Traffic Drop Alert\n\n```\nchart_id: 123\nname: \"Traffic Drop\"\nalert_type: \"relative\"\noperator: \"decreases_by\"\nvalue: 50\nquery_period: 3600\nconfirmation_period: 300\n```\n\n### Anomaly Detection\n\n```\nchart_id: 123\nname: \"Unusual Activity\"\nalert_type: \"anomaly_rrcf\"\noperator: \"higher_than\"\nanomaly_sensitivity: 5\nshown_interval: 10800\nanomaly_trigger: \"any\"\n```\n\nFor anomaly alerts the direction comes from `anomaly_trigger` (`any` = both directions), not from `operator`/`value` (threshold parameters).\n\n## Common Mistakes\n\n| Mistake | Correct Approach |\n| -------------------------------------------- | ---------------------------------------------- |\n| Using relative operators with threshold | `increases_by` only works with `relative` type |\n| Guessing at a significance or threshold | Query the chart's history first, then quote how often it would fire |\n| No confirmation period for noisy metrics | Add 60-300s to prevent flapping |\n| Alerting on all series when only some matter | Specify `series_names` array |\n\n## Managing Alerts\n\n| Action | Tool |\n| ------------------ | -------------------------- |\n| List all alerts | `chart_alerts` |\n| Get alert details | `chart_alert` |\n| Create | `create_chart_alert` |\n| Edit configuration | `edit_chart_alert` |\n| Pause/unpause | `toggle_chart_alert_pause` |\n| Delete | `delete_chart_alert` |\n""---\nname: chart-alerts\ndescription: Guide for creating and managing Better Stack chart alerts. Load when asked to set up alerting, create alerts on charts, or configure monitoring thresholds.\n---\n\n# Chart Alerts Guide\n\nChart alerts monitor metrics and logs queries and trigger incidents when conditions are met. They are attached to charts on dashboards or explorations. `chart_alert_help` has the server's current reference.\n\n> **Two-tier thresholds:** a chart alert holds a single `value`. To reproduce a Datadog warning + critical monitor, create **two** alerts on the same chart (one per threshold).\n\n## Alert Types\n\n| Type | Use Case | Key Parameters |\n| -------------- | ------------------------------------------------------------ | ---------------------------------------------------------- |\n| `threshold` | Known fixed limit exceeded (disk > 90%, error rate > 5%) | `value`, `operator` |\n| `relative` | Change in % vs. the past, when the user asks for one | `value`, `operator`, `query_period` |\n| `anomaly_rrcf` | Unusual patterns (ML-based), with a floor by default (below) | `shown_interval`, `anomaly_sensitivity`, `anomaly_trigger` |\n\n## What to Alert On\n\nUnless the user asks otherwise, every alert fires only on something both bad and unusual:\n\n- **Outcomes, not activity**: orders completed, jobs finished, age of the oldest queued job. A stuck\n worker that retries every 2 s keeps emitting spans and requests while nothing gets done.\n- **Saturation** (memory, disk, connection pools): `threshold`, since it is bad at the same level every day.\n- **Customer impact** (error rate, latency, failed checkouts, orders dropping): `threshold`; for a\n drop, `lower_than` a value under the quietest normal hour. An anomaly or relative alert learns a\n sustained change as the new normal and resolves within about an hour while the outage goes on.\n- **Anything else unusual**: `anomaly_rrcf` with a floor.\n- **Slices**: group error and latency charts by endpoint or error type. Each series is checked on its\n own, so a failure in one slice crosses the bar instead of hiding in the whole-traffic average.\n- **Low-traffic services** (a few requests per second or fewer): a ratio jumps on a handful of\n requests; alert on error counts instead.\n\n## Operators\n\n| Operator | Description | Alert Types |\n| ---------------------- | -------------------------------------- | ------------------- |\n| `higher_than` | Value exceeds threshold | threshold, relative |\n| `lower_than` | Value below threshold | threshold, relative |\n| `higher_than_or_equal` | Value >= threshold | threshold, relative |\n| `lower_than_or_equal` | Value <= threshold | threshold, relative |\n| `equal` | Value equals threshold | threshold, relative |\n| `increases_by` | Value increases by X% | relative only |\n| `decreases_by` | Value decreases by X% | relative only |\n| `changes_by` | Value changes by X% (either direction) | relative only |\n\n## Chart Requirements\n\nOnly certain chart types support alerts:\n\n| Chart Type | Requirements |\n| -------------- | ---------------------------------------------- |\n| `line_chart` | Must be timeseries, query must have `{{time}}` |\n| `bar_chart` | Must be timeseries, query must have `{{time}}` |\n| `number_chart` | Query must have `{{time}}` |\n| `tail_chart` | No additional requirements |\n\n**Prefer line charts on dashboards** for alerts - they provide the best visibility into metric trends and make it easy to correlate alerts with visual data.\n\n## Key Parameters\n\n### Value (Threshold/Relative)\n\n**Critical**: The `value` is compared with the raw numbers the chart query returns, not with what the chart displays.\n\n| Chart | Example Query | Alert Value for \"85%\" |\n| ---------------------------------------------- | --------------------------------------------------------------- | --------------------- |\n| `unit: \"percent\"` | `sumIf(logs_count, label('level') = 'error') / sum(logs_count)` | `0.85` |\n| Data already 0-100, without the `percent` unit | `avg(cpu_percent)` | `85` |\n\nA `percent` chart holds 0-1 ratios and multiplies them by 100 for display, so it shows 0.85 as 85% while its alert `value` stays `0.85`. The exception is a `stack_to_100` bar chart: it sets `percent` itself but its query returns raw values, so its thresholds are raw values too. A relative alert's `value` stays a percentage change either way (50 means 50%).\n\n**Before creating an alert**, check the chart's `unit` and actual values using `chart`.\n\n### Confirmation Period\n\nHow long the condition must persist before triggering an alert. Fixed buckets (seconds): `0, 30, 60, 120, 300, 600, 1800, 3600, 7200, 14400`.\n\n### Recovery Period\n\nHow long the condition must be resolved before the incident is closed. Same fixed buckets as confirmation period, or `null` = no auto-recovery.\n\n- Set equal to or higher than confirmation_period\n- Prevents alert flapping\n\n### Missing Data\n\n`on_missing_data` is what a threshold or relative alert does when its chart returns no data: `dont_fire`\n(default), `treat_as_zero`, `treat_as_previous`, or `start_incident`. Use `start_incident` on throughput and\noutcome charts (requests, orders completed): when that data stops, the service is down, and `dont_fire`\nwould resolve any open incident and stay quiet. On a sparse chart (a few orders an hour), empty buckets\nin quiet hours open incidents too: widen `aggregation_interval` there, or keep `dont_fire`. Anomaly\nalerts take it from the chart and refuse it.\n\n### Query Period (Relative Alerts)\n\nHow far back to look for comparison. Fixed buckets (seconds): `30, 60, 120, 300, 600, 1800, 3600` — **capped at 1 hour**; \"vs 1 day/week ago\" comparisons are not possible.\n\n### Anomaly Settings\n\n| Parameter | Range | Description |\n| --------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------- |\n| `anomaly_sensitivity` | 1-10 | Counter to its name: HIGHER = fewer alerts, LOWER = more sensitive (default: 5). Shown as **Significance** in the sidebar |\n| `anomaly_trigger` | `any`, `higher`, `lower` | Which anomalies to alert on |\n| `shown_interval` | seconds | Aggregation interval; the training window spans ~10× this (default: 10800 = 3h) |\n\nDon't guess a significance or a threshold: query the chart's data over the past days first (`chart`, then `query`) and say how often the condition would have been met.\n\n## Extra Conditions: Unusual AND Bad\n\n`additional_conditions` ANDs up to 4 more conditions onto the alert's own: it fires only when every\ncondition breaches in the same time bucket, on any of the series it covers, not necessarily the same\none. On a grouped chart, an unscoped floor is met by whichever series crosses it. Each is `{alert_type, operator, value, series_names,\nseries_names_except}`, `threshold` or `relative` only (anomaly detection can be the alert's own type,\nnot an extra condition), and each may scope its own series.\n\n**Give every anomaly alert a floor by default.** On its own an anomaly alert fires on anything\nunusual, including harmless shifts: after a node replacement, CPU moving to 21% or a p95 of 76 ms is\nunusual but fine. A threshold condition such as \"and CPU > 80\" or \"and p95 > 500 ms\" keeps it to what\nis both unusual and bad. Pick the floor from the chart's real values (`chart`), the way you would a\nthreshold alert's `value`.\n\nOn `edit_chart_alert` the list replaces the alert's conditions as a whole: omit it to keep them,\npass `[]` to remove them. `chart_alert` lists them as **Additional Conditions**.\n\nCheck a floor the way you would a threshold: query the chart's data over the past days (`chart`, then `query`) and count how often it crosses the floor. The alert fires at most where the metric is both unusual and past the floor.\n\n## Metadata\n\n`metadata` tags the alert, e.g. `{severity: \"critical\", owner: [\"payments\"]}`: each value a string or\na list of strings. The incidents the alert opens carry them. On `edit_chart_alert` it replaces the\nalert's metadata as a whole; `{}` removes it.\n\n## Series Names\n\nMonitor specific series or all:\n\n- Empty array `[]` = monitor all series\n- `[\"production\", \"staging\"]` = only these series\n- Use when you want to alert on specific hosts/services\n\n## Escalation Targets\n\n| Value | Description |\n| -------------------------- | --------------------------------------------------------- |\n| `current_team` | Default on-call for the team |\n| `gid://uptime/Policy/<id>` | A specific escalation policy (from `escalation_policies`) |\n| `gid://uptime/Team/<id>` | A specific team's default on-call |\n\nTo match a Datadog monitor's notification routing, call `escalation_policies` and pass the closest policy's gid.\n\n## Notification Channels\n\n| Channel | Default | Use Case |\n| ------- | ------- | -------------------- |\n| `email` | `true` | Always on |\n| `push` | `false` | Mobile notifications |\n| `sms` | `false` | Urgent issues |\n| `call` | `false` | Critical outages |\n\nUnless the user asks otherwise, call, SMS and a `critical` severity are for customer impact only. An\nalert that the chart's history shows firing at the same time every day (a batch job, a backup, a deploy) is a\nknown event: raise its threshold or floor above it.\n\n## Process\n\nFind or create the chart (`dashboard` / `add_chart_to_dashboard`); an error or latency chart is grouped by endpoint or error type, and on a low-traffic service it counts errors → verify it supports alerts and check its value range (`chart`, then `query` over a few days) → choose the alert type → `create_chart_alert` → tell the user how often each alert's conditions were met over the past week (`query`).\n\n## Examples\n\n### Disk Almost Full Alert\n\n```\nchart_id: 123\nname: \"Disk almost full\"\nalert_type: \"threshold\"\noperator: \"higher_than\"\nvalue: 0.90 # On a `percent` chart (0-1 data shown as 90%); 90 only if the query returns 0-100\nconfirmation_period: 60\n```\n\n### Orders Stopped Alert\n\n```\nchart_id: 123\nname: \"Orders stopped\"\nalert_type: \"threshold\"\noperator: \"lower_than\"\nvalue: 5 # Orders per minute, under the quietest normal hour\nconfirmation_period: 600\n```\n\n### Anomaly Detection with a Floor\n\n```\nchart_id: 123\nname: \"CPU unusually high\"\nalert_type: \"anomaly_rrcf\"\noperator: \"higher_than\"\nanomaly_sensitivity: 5\nshown_interval: 10800\nanomaly_trigger: \"higher\"\nadditional_conditions: [{ alert_type: \"threshold\", operator: \"higher_than\", value: 0.8 }] # 80 only if the query returns 0-100\nmetadata: { severity: \"warning\" }\n```\n\nFor anomaly alerts the direction comes from `anomaly_trigger` (`any` = both directions), not from `operator`/`value` (threshold parameters).\n\n## Common Mistakes\n\n| Mistake | Correct Approach |\n| -------------------------------------------- | ---------------------------------------------------------------- |\n| Using relative operators with threshold | `increases_by` only works with `relative` type |\n| `value: 85` for 85% on a `percent` chart | Its data is 0-1: use `0.85` (the chart and alert show it as 85%) |\n| Guessing at a significance or threshold | Query the chart's history first, then quote how often it would fire |\n| No confirmation period for noisy metrics | Add 60-300s to prevent flapping |\n| Alerting on all series when only some matter | Specify `series_names` array |\n| Anomaly alert with no floor | Add a threshold in `additional_conditions` |\n\n## Managing Alerts\n\n| Action | Tool |\n| ------------------ | -------------------------- |\n| List all alerts | `chart_alerts` |\n| Get alert details | `chart_alert` |\n| Create | `create_chart_alert` |\n| Edit configuration | `edit_chart_alert` |\n| Pause | `pause_chart_alert` |\n| Unpause | `unpause_chart_alert` |\n| Delete | `delete_chart_alert` |\n"Full snapshot data
{
"description": "Guide for creating and managing Better Stack chart alerts. Load when asked to set up alerting, create alerts on charts, or configure monitoring thresholds.",
"included_files": [],
"name": "chart-alerts",
"skill_md_contents": "---\nname: chart-alerts\ndescription: Guide for creating and managing Better Stack chart alerts. Load when asked to set up alerting, create alerts on charts, or configure monitoring thresholds.\n---\n\n# Chart Alerts Guide\n\nChart alerts monitor metrics and logs queries and trigger incidents when conditions are met. They are attached to charts on dashboards or explorations. `chart_alert_help` has the server's current reference.\n\n> **Two-tier thresholds:** a chart alert holds a single `value`. To reproduce a Datadog warning + critical monitor, create **two** alerts on the same chart (one per threshold).\n\n## Alert Types\n\n| Type | Use Case | Key Parameters |\n| -------------- | ------------------------------------------------------------ | ---------------------------------------------------------- |\n| `threshold` | Known fixed limit exceeded (disk > 90%, error rate > 5%) | `value`, `operator` |\n| `relative` | Change in % vs. the past, when the user asks for one | `value`, `operator`, `query_period` |\n| `anomaly_rrcf` | Unusual patterns (ML-based), with a floor by default (below) | `shown_interval`, `anomaly_sensitivity`, `anomaly_trigger` |\n\n## What to Alert On\n\nUnless the user asks otherwise, every alert fires only on something both bad and unusual:\n\n- **Outcomes, not activity**: orders completed, jobs finished, age of the oldest queued job. A stuck\n worker that retries every 2 s keeps emitting spans and requests while nothing gets done.\n- **Saturation** (memory, disk, connection pools): `threshold`, since it is bad at the same level every day.\n- **Customer impact** (error rate, latency, failed checkouts, orders dropping): `threshold`; for a\n drop, `lower_than` a value under the quietest normal hour. An anomaly or relative alert learns a\n sustained change as the new normal and resolves within about an hour while the outage goes on.\n- **Anything else unusual**: `anomaly_rrcf` with a floor.\n- **Slices**: group error and latency charts by endpoint or error type. Each series is checked on its\n own, so a failure in one slice crosses the bar instead of hiding in the whole-traffic average.\n- **Low-traffic services** (a few requests per second or fewer): a ratio jumps on a handful of\n requests; alert on error counts instead.\n\n## Operators\n\n| Operator | Description | Alert Types |\n| ---------------------- | -------------------------------------- | ------------------- |\n| `higher_than` | Value exceeds threshold | threshold, relative |\n| `lower_than` | Value below threshold | threshold, relative |\n| `higher_than_or_equal` | Value >= threshold | threshold, relative |\n| `lower_than_or_equal` | Value <= threshold | threshold, relative |\n| `equal` | Value equals threshold | threshold, relative |\n| `increases_by` | Value increases by X% | relative only |\n| `decreases_by` | Value decreases by X% | relative only |\n| `changes_by` | Value changes by X% (either direction) | relative only |\n\n## Chart Requirements\n\nOnly certain chart types support alerts:\n\n| Chart Type | Requirements |\n| -------------- | ---------------------------------------------- |\n| `line_chart` | Must be timeseries, query must have `{{time}}` |\n| `bar_chart` | Must be timeseries, query must have `{{time}}` |\n| `number_chart` | Query must have `{{time}}` |\n| `tail_chart` | No additional requirements |\n\n**Prefer line charts on dashboards** for alerts - they provide the best visibility into metric trends and make it easy to correlate alerts with visual data.\n\n## Key Parameters\n\n### Value (Threshold/Relative)\n\n**Critical**: The `value` is compared with the raw numbers the chart query returns, not with what the chart displays.\n\n| Chart | Example Query | Alert Value for \"85%\" |\n| ---------------------------------------------- | --------------------------------------------------------------- | --------------------- |\n| `unit: \"percent\"` | `sumIf(logs_count, label('level') = 'error') / sum(logs_count)` | `0.85` |\n| Data already 0-100, without the `percent` unit | `avg(cpu_percent)` | `85` |\n\nA `percent` chart holds 0-1 ratios and multiplies them by 100 for display, so it shows 0.85 as 85% while its alert `value` stays `0.85`. The exception is a `stack_to_100` bar chart: it sets `percent` itself but its query returns raw values, so its thresholds are raw values too. A relative alert's `value` stays a percentage change either way (50 means 50%).\n\n**Before creating an alert**, check the chart's `unit` and actual values using `chart`.\n\n### Confirmation Period\n\nHow long the condition must persist before triggering an alert. Fixed buckets (seconds): `0, 30, 60, 120, 300, 600, 1800, 3600, 7200, 14400`.\n\n### Recovery Period\n\nHow long the condition must be resolved before the incident is closed. Same fixed buckets as confirmation period, or `null` = no auto-recovery.\n\n- Set equal to or higher than confirmation_period\n- Prevents alert flapping\n\n### Missing Data\n\n`on_missing_data` is what a threshold or relative alert does when its chart returns no data: `dont_fire`\n(default), `treat_as_zero`, `treat_as_previous`, or `start_incident`. Use `start_incident` on throughput and\noutcome charts (requests, orders completed): when that data stops, the service is down, and `dont_fire`\nwould resolve any open incident and stay quiet. On a sparse chart (a few orders an hour), empty buckets\nin quiet hours open incidents too: widen `aggregation_interval` there, or keep `dont_fire`. Anomaly\nalerts take it from the chart and refuse it.\n\n### Query Period (Relative Alerts)\n\nHow far back to look for comparison. Fixed buckets (seconds): `30, 60, 120, 300, 600, 1800, 3600` — **capped at 1 hour**; \"vs 1 day/week ago\" comparisons are not possible.\n\n### Anomaly Settings\n\n| Parameter | Range | Description |\n| --------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------- |\n| `anomaly_sensitivity` | 1-10 | Counter to its name: HIGHER = fewer alerts, LOWER = more sensitive (default: 5). Shown as **Significance** in the sidebar |\n| `anomaly_trigger` | `any`, `higher`, `lower` | Which anomalies to alert on |\n| `shown_interval` | seconds | Aggregation interval; the training window spans ~10× this (default: 10800 = 3h) |\n\nDon't guess a significance or a threshold: query the chart's data over the past days first (`chart`, then `query`) and say how often the condition would have been met.\n\n## Extra Conditions: Unusual AND Bad\n\n`additional_conditions` ANDs up to 4 more conditions onto the alert's own: it fires only when every\ncondition breaches in the same time bucket, on any of the series it covers, not necessarily the same\none. On a grouped chart, an unscoped floor is met by whichever series crosses it. Each is `{alert_type, operator, value, series_names,\nseries_names_except}`, `threshold` or `relative` only (anomaly detection can be the alert's own type,\nnot an extra condition), and each may scope its own series.\n\n**Give every anomaly alert a floor by default.** On its own an anomaly alert fires on anything\nunusual, including harmless shifts: after a node replacement, CPU moving to 21% or a p95 of 76 ms is\nunusual but fine. A threshold condition such as \"and CPU > 80\" or \"and p95 > 500 ms\" keeps it to what\nis both unusual and bad. Pick the floor from the chart's real values (`chart`), the way you would a\nthreshold alert's `value`.\n\nOn `edit_chart_alert` the list replaces the alert's conditions as a whole: omit it to keep them,\npass `[]` to remove them. `chart_alert` lists them as **Additional Conditions**.\n\nCheck a floor the way you would a threshold: query the chart's data over the past days (`chart`, then `query`) and count how often it crosses the floor. The alert fires at most where the metric is both unusual and past the floor.\n\n## Metadata\n\n`metadata` tags the alert, e.g. `{severity: \"critical\", owner: [\"payments\"]}`: each value a string or\na list of strings. The incidents the alert opens carry them. On `edit_chart_alert` it replaces the\nalert's metadata as a whole; `{}` removes it.\n\n## Series Names\n\nMonitor specific series or all:\n\n- Empty array `[]` = monitor all series\n- `[\"production\", \"staging\"]` = only these series\n- Use when you want to alert on specific hosts/services\n\n## Escalation Targets\n\n| Value | Description |\n| -------------------------- | --------------------------------------------------------- |\n| `current_team` | Default on-call for the team |\n| `gid://uptime/Policy/<id>` | A specific escalation policy (from `escalation_policies`) |\n| `gid://uptime/Team/<id>` | A specific team's default on-call |\n\nTo match a Datadog monitor's notification routing, call `escalation_policies` and pass the closest policy's gid.\n\n## Notification Channels\n\n| Channel | Default | Use Case |\n| ------- | ------- | -------------------- |\n| `email` | `true` | Always on |\n| `push` | `false` | Mobile notifications |\n| `sms` | `false` | Urgent issues |\n| `call` | `false` | Critical outages |\n\nUnless the user asks otherwise, call, SMS and a `critical` severity are for customer impact only. An\nalert that the chart's history shows firing at the same time every day (a batch job, a backup, a deploy) is a\nknown event: raise its threshold or floor above it.\n\n## Process\n\nFind or create the chart (`dashboard` / `add_chart_to_dashboard`); an error or latency chart is grouped by endpoint or error type, and on a low-traffic service it counts errors → verify it supports alerts and check its value range (`chart`, then `query` over a few days) → choose the alert type → `create_chart_alert` → tell the user how often each alert's conditions were met over the past week (`query`).\n\n## Examples\n\n### Disk Almost Full Alert\n\n```\nchart_id: 123\nname: \"Disk almost full\"\nalert_type: \"threshold\"\noperator: \"higher_than\"\nvalue: 0.90 # On a `percent` chart (0-1 data shown as 90%); 90 only if the query returns 0-100\nconfirmation_period: 60\n```\n\n### Orders Stopped Alert\n\n```\nchart_id: 123\nname: \"Orders stopped\"\nalert_type: \"threshold\"\noperator: \"lower_than\"\nvalue: 5 # Orders per minute, under the quietest normal hour\nconfirmation_period: 600\n```\n\n### Anomaly Detection with a Floor\n\n```\nchart_id: 123\nname: \"CPU unusually high\"\nalert_type: \"anomaly_rrcf\"\noperator: \"higher_than\"\nanomaly_sensitivity: 5\nshown_interval: 10800\nanomaly_trigger: \"higher\"\nadditional_conditions: [{ alert_type: \"threshold\", operator: \"higher_than\", value: 0.8 }] # 80 only if the query returns 0-100\nmetadata: { severity: \"warning\" }\n```\n\nFor anomaly alerts the direction comes from `anomaly_trigger` (`any` = both directions), not from `operator`/`value` (threshold parameters).\n\n## Common Mistakes\n\n| Mistake | Correct Approach |\n| -------------------------------------------- | ---------------------------------------------------------------- |\n| Using relative operators with threshold | `increases_by` only works with `relative` type |\n| `value: 85` for 85% on a `percent` chart | Its data is 0-1: use `0.85` (the chart and alert show it as 85%) |\n| Guessing at a significance or threshold | Query the chart's history first, then quote how often it would fire |\n| No confirmation period for noisy metrics | Add 60-300s to prevent flapping |\n| Alerting on all series when only some matter | Specify `series_names` array |\n| Anomaly alert with no floor | Add a threshold in `additional_conditions` |\n\n## Managing Alerts\n\n| Action | Tool |\n| ------------------ | -------------------------- |\n| List all alerts | `chart_alerts` |\n| Get alert details | `chart_alert` |\n| Create | `create_chart_alert` |\n| Edit configuration | `edit_chart_alert` |\n| Pause | `pause_chart_alert` |\n| Unpause | `unpause_chart_alert` |\n| Delete | `delete_chart_alert` |\n"
}SHA-256 of public snapshot: 8cce8af53061bdc978a957519be5e98d78223d13de2eadcb83ec3d4acdf335ad