← Plugin catalog
Developer Tools

Honeycomb

honeycomb.io v1.1.0

Analyze observability data

Create a free account to save this plugin.

From the developer

Not specified in the available declarations: capabilities, limitations, pricing & account requirements.

Onboarding example

Which of my SLOs are burning, and why?
Sources & dates

Developer declarations; not independently tested. Listing last checked Nov 22, 2026. Package observed Nov 21, 2026.

View package source →

Description

From the marketplace listing

Honeycomb helps users investigate observability data, inspect traces, AI conversations, and anomaly signals, run queries and BubbleUp analyses, manage alerts and dashboards, and follow instrumentation migration guidance through ChatGPT.

Language: English · Automatically detected from descriptions.

What changed

· 3 observations

Positioning & listing

Full description changed from “traces and AI conversations, run queries and BubbleUp analyses, and manage boards, SLOs, triggers, markers, and notification recipients” to “traces, AI conversations, and anomaly signals, run queries and BubbleUp analyses, manage alerts and dashboards, and follow instrumentation...

View comparison →

Technical updates · 1

Package or technical metadata updated

View technical comparison →

Similar plugins

Finding similar plugins… Results will appear here automatically.

Browse plugins →

Files & skills

File archives

Plugin package52 files · 124 KBBrowse files →
beeline-migration3 files · 3.72 KBBrowse files →
create-honeycomb-board3 files · 6.97 KBBrowse files →
metrics-queries4 files · 10.1 KBBrowse files →
observability-fundamentals2 files · 5.63 KBBrowse files →
otel-genai-instrumentation8 files · 25 KBBrowse files →
otel-instrumentation9 files · 32.2 KBBrowse files →
otel-migration6 files · 15.7 KBBrowse files →
production-investigation4 files · 8.79 KBBrowse files →
query-patterns6 files · 13.7 KBBrowse files →
slos-and-triggers4 files · 6.93 KBBrowse files →
verify-recent-trace1 files · 1.39 KBBrowse files →
Skill instructions
beeline-migration4.77 KB

View saved version →

---
name: beeline-migration
description: >
  Step-by-step guide for migrating from Honeycomb Beelines (End of Life) to
  OpenTelemetry instrumentation.
  Trigger phrases: "migrate from Beelines",
  "upgrade from Beeline to OpenTelemetry", "migrate to OTel", "replace Beelines",
  "Beeline end of life", "Beeline EOL", "switch from Beeline to OTel",
  "migrate Go Beeline", "migrate Python Beeline", "migrate Node Beeline",
  "migrate Java Beeline", "migrate Ruby Beeline", "W3C trace headers",
  "W3C propagation", "incremental migration to OpenTelemetry",
  or any request about migrating from Honeycomb Beelines to OpenTelemetry SDKs.
metadata:
  version: "1.0.0"
---

# Beeline to OpenTelemetry Migration

Step-by-step guide for migrating from Honeycomb Beelines (now End of Life)
to OpenTelemetry instrumentation.

## Status

Honeycomb Beelines have reached **End of Life** and are **archived**. All new
instrumentation should use OpenTelemetry. Existing Beeline users should migrate
as soon as practical.

## Migration Strategy

Migration follows a two-phase approach that allows incremental, service-by-service
migration without breaking distributed traces.

### Phase 1: Enable W3C Trace Propagation (All Services)

Before migrating any service to OTel, **all** services must support W3C trace
headers. This enables Beeline and OTel services to share trace context.

1. Upgrade each Beeline to the minimum version supporting W3C headers
2. Configure each Beeline to use W3C propagation format
3. Deploy all services with W3C enabled
4. **Verify**: Traces still link correctly across services

**Minimum Beeline versions for W3C support:**

| Language | Minimum Version |
|----------|----------------|
| Go | 1.4.0 |
| Java | 1.7.0 |
| Node.js | 3.2.2 |
| Python | 2.18.0 |
| Ruby | 2.8.0 |

### Phase 2: Migrate Each Service to OTel (One at a Time)

After all services support W3C headers:

1. Choose a service to migrate (start with leaf services — fewest dependencies)
2. Replace Beeline SDK with OpenTelemetry SDK
3. Configure OTLP exporter to point to Honeycomb
4. Add auto-instrumentation libraries
5. Replicate any custom Beeline instrumentation in OTel
6. Deploy and verify traces still connect
7. Repeat for next service

**Key rule**: Complete Phase 1 across ALL services before starting Phase 2 on ANY service.

## W3C Propagation Configuration

### Go Beeline
```go
beeline.Init(beeline.Config{
    HTTPPropagationHook: propagation.W3C,
})
```

### Python Beeline
```python
beeline.init(
    http_trace_propagation_hook=beeline.propagation.w3c.http_trace_propagation_hook,
    http_trace_parser_hook=beeline.propagation.w3c.http_trace_parser_hook,
)
```

### Node.js Beeline
```javascript
const beeline = require("honeycomb-beeline")({
    httpTraceParserHook: beeline.w3c.httpTraceParserHook,
    httpTracePropagationHook: beeline.w3c.httpTracePropagationHook,
});
```

For Java and Ruby configurations, consult `${CLAUDE_PLUGIN_ROOT}/skills/beeline-migration/references/w3c-propagation.md`.

## Service Migration Checklist

For each service being migrated from Beeline to OTel:

- [ ] Beeline version supports W3C (Phase 1 complete)
- [ ] Install OTel SDK and OTLP exporter packages
- [ ] Configure OTLP endpoint and headers for Honeycomb
- [ ] Set `OTEL_SERVICE_NAME` to match existing service name
- [ ] Add auto-instrumentation libraries (HTTP, DB, etc.)
- [ ] Port custom spans: Beeline `startSpan()` -> OTel `tracer.start_span()`
- [ ] Port custom attributes: Beeline `addField()` -> OTel `span.set_attribute()`
- [ ] Remove Beeline dependency
- [ ] Deploy and verify: traces link across Beeline and OTel services
- [ ] Verify: custom attributes appear in Honeycomb

## Migration Safety Checklist

- **Complete Phase 1 across all services before starting Phase 2** — mixed propagation formats break trace linking across service boundaries
- **Keep `OTEL_SERVICE_NAME` identical to the Beeline service name** — Honeycomb uses this as the dataset name, and changing it splits y

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 2 — file references are metadata, not package contents.

create-honeycomb-board7.23 KB

View saved version →

---
name: create-honeycomb-board
description: >
  Design and then create a board (dashboard) in Honeycomb with queries and SLOs.
  Trigger phrases: "create a board", "make a board", "build a dashboard",
  "create a Honeycomb board", "make a dashboard in Honeycomb",
  "set up a board", "dashboard for my service", "visualize service health",
  "golden signals dashboard", "set up monitoring board",
  or any request to design and create or build a Honeycomb board or dashboard.
metadata:
  version: "1.0.0"
allowed-tools:
  - mcp__honeycomb__create_board
  - mcp__honeycomb__run_query
  - mcp__honeycomb__get_query_results
  - mcp__honeycomb__get_workspace_context
  - mcp__honeycomb__get_environment
  - mcp__honeycomb__get_dataset
  - mcp__honeycomb__get_dataset_columns
  - mcp__honeycomb__find_columns
  - mcp__honeycomb__find_queries
  - mcp__honeycomb__get_slos
  - mcp__honeycomb__get_triggers
  - mcp__honeycomb__list_boards
  - Read
  - Grep
  - Glob
  - AskUserQuestion
---

# Create a Honeycomb Board

Build a board (dashboard) in Honeycomb using the `create_board` MCP tool.
There is no update tool — define it well before creating.

When building a board, think about the purpose and time frame involved. Some examples:

- a board for a service. This should be timeless, looking at the service's health, performance, and business metrics. Do not do any problem diagnosis or investigation when building this board. Do not express opinions or summarize graphs in text panels. The board should be a representation of the service's health at whatever moment someone looks at it.
- a board for a feature. This should look at the feature's usage trends, and its impact on the business. This board would have a time frame of 7 days, and would not include any infrastructure metrics or service dependencies.
- a board for a problem. This might be created during an incident, or afterward. This one would have a time frame specific to the incident. It would include investigations, and your opinions about what is happening. Patterns of what to look for are appropriate here.

## Workflow

### Gather SLOs

Use `get_slos` to list SLOs in the environment. Relevant SLOs will go on the board as `slo` panels.

### Gather descriptive context

Look at the code and docs (using Read/Grep/Glob) to understand the service or feature. Use this to write a text panel. Link to GitHub or documentation if you can.

This will vary greatly depending on the purpose of the board.

Description of the application and link to the code - great for a service board.

What is the feature, and what business impact does it have? - great for a feature board.

What is the problem, and what is the impact? What patterns do we see? - great for a problem board.

### Build candidate queries

**Get Honeycomb context**: Call `get_workspace_context` for available environments. Use `get_environment` to find datasets — each dataset corresponds to an `OTEL_SERVICE_NAME`.

**Code context**: Look at the language and any custom attributes (often prefixed `app.`). Custom fields are prime candidates for breakdowns and business metrics.

**Discover columns**: Use `find_columns` or `get_dataset_columns`. Pay special attention to non-standard columns — those are specific to the application.

**Find queries**: Use `find_queries` to see what people have already queried. Check `get_triggers` for what they alert on — those indicate what matters.

**Time range**: Default to 2 hours. Use 8–24 hours for lower-volume services. Use 7 days for feature usage boards. Keep it consistent across all panels.

Aim for 6–12 graphs. Stat panels are bonus — they don't count against this limit.

List your candidate queries for the user with reasons before running them. See `${CLAUDE_PLUGIN_ROOT}/skills/create-honeycomb-board/references/board-queries.md` for what to include and how to write each kind.

### Run and check queries

Use `run_query` for each candidate query. Each result returns a query run PK (like `QR-abc123`) — you'll

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 2 — file references are metadata, not package contents.

metrics-queries13.4 KB

View saved version →

---
name: metrics-queries
description: >
  How to query OpenTelemetry metrics datasets in Honeycomb correctly. Metrics
  datasets follow different rules from trace/event datasets — many operations
  (bare COUNT, RATE_SUM, RATE_AVG, RATE_MAX, CONCURRENCY) are forbidden,
  temporal aggregation is automatic, and each metric has its own attributes.
  Use this skill when querying a metrics dataset (gauges, counters, histograms,
  sums), asking about temporal aggregation (RATE, INCREASE, SUMMARIZE, LAST),
  finding the metrics dataset or discovering metric names and attributes,
  debugging unexpected metrics query results, or querying infrastructure
  metrics like CPU, memory, disk I/O, or network stats. Do NOT use for
  instrumenting metrics (use otel-instrumentation), querying event datasets
  with "metrics" in their name, or conceptual questions (use
  observability-fundamentals).
metadata:
  version: "1.0.0"
---

# Querying Metrics in Honeycomb

Metrics datasets in Honeycomb behave differently from tracing/event datasets.
Operations that work on traces may fail or produce misleading results on metrics.
This skill covers those differences so you construct correct, useful metrics queries.

## Finding the Metrics Dataset

Metrics datasets are **not** identified by having "metrics" in their name. Many event
datasets contain "metrics" in their slug (e.g., `kafka-metrics`, `refinery-metrics`,
`kubernetes-node-metrics`). These are ordinary event datasets, not metrics datasets.

**How to identify the real metrics dataset:**

1. Call `get_environment` and look for rows where `dataset_type` = **`metrics`**.
   The slug is typically `metrics` but may differ per environment.
2. Alternatively, call `get_dataset_columns` on a candidate dataset — metrics datasets
   return a **`MetricInfo`** column showing type metadata like `gauge`,
   `sum(cumulative,monotonic)`, or `histogram(delta)`. Event datasets do not have this.

**Do not guess the dataset.** Always verify via `get_environment` or `get_dataset_columns`
before constructing a metrics query. If the user says "metrics" but means an event dataset
with metrics-like fields (e.g., `telegraf`, `system_stats`), the query rules below do not apply —
those are event datasets and follow normal query patterns from the **query-patterns** skill.

## Discovering Metrics and Their Attributes

Metrics datasets have a fundamentally different schema from event datasets. Each metric
has its own set of resource and data point attributes. Two metrics in the same dataset
may have completely different attributes available for filtering and grouping.

**Workflow for discovering what to query:**

1. **Find metric names:** Call `get_dataset_columns` on the metrics dataset (without
   `metric_name`). This returns metric names with their types in `MetricInfo`.
   Use `find_columns` with keywords to search for specific metrics (e.g., "cpu", "memory",
   "http request duration").

2. **Find attributes for a specific metric:** Call `get_dataset_columns` with the
   `metric_name` parameter set to the metric you want to query (e.g.,
   `metric_name: "k8s.pod.memory.usage"`). This returns the resource attributes and
   data point attributes that co-occur with that metric, along with sample values.
   These are what you can use in WHERE and GROUP BY clauses.

3. **Validate before querying:** Not all attributes exist on all metrics. Always use
   step 2 to confirm available attributes before adding them to filters or breakdowns.

## Allowed vs. Forbidden Operations on Metrics Datasets

The following operations are **NOT allowed** on metrics datasets:

| Forbidden Operation | Why |
|---------------------|-----|
| `COUNT` (without column) | Counts metric events, not metric values — meaningless for metrics |
| `RATE_SUM` | Not supported on metrics datasets |
| `RATE_AVG` | Not supported on metrics datasets |
| `RATE_MAX` | Not supported on metrics datasets |
| `CONCURRENCY` | Requires span duration; metrics have no duration |

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 3 — file references are metadata, not package contents.

observability-fundamentals6.77 KB

View saved version →

---
name: observability-fundamentals
description: >
  First principles behind observability — wide events, high cardinality, the core
  analysis loop, events vs metrics vs logs, and how instrumentation connects to
  debugging outcomes. Grounds recommendations in first principles rather than
  tool-specific how-to.
  Trigger phrases: "what is observability", "why observability", "why Honeycomb",
  "events vs metrics vs logs", "events vs metrics", "events vs logs",
  "metrics vs logs", "why wide events", "what is high cardinality",
  "core analysis loop", "observability vs monitoring", "what is dimensionality",
  "explain observability", or any conceptual question about observability
  or why Honeycomb's approach differs from traditional monitoring.
metadata:
  version: "1.0.0"
---

# Observability Fundamentals

First principles behind Honeycomb's approach to observability. Use this to ground
recommendations and answer conceptual questions — for SDK setup and tool-specific
guidance, see the **otel-instrumentation** and **query-patterns** skills.

## Definitions

**Observability**: The ability to understand and explain any state your system can
get into, no matter how novel or complex — by examining what the system produces,
without deploying new code for each new question.

**Wide event**: A flat key-value record capturing the full context of a unit of work —
who made the request, which endpoint, cache hit/miss, build version, duration, error
status, and any business context relevant to the operation. In OpenTelemetry, a **span**
is a wide event.

**High cardinality**: The number of unique values a field can have. `user.id` with
millions of values is high cardinality. `http.method` with a handful is low cardinality.

**High dimensionality**: The number of distinct fields on your events. A span with
50 attributes has high dimensionality.

| Concept | Observability | Traditional Monitoring |
|---|---|---|
| Questions | Arbitrary, unknown ahead of time | Pre-defined (dashboards, alerts) |
| Data shape | Decided at query time | Decided at instrumentation time |
| Cardinality | High cardinality is valuable | High cardinality is expensive |
| Investigation | Explore → narrow → confirm | Check dashboard → escalate |

## Why Wide Events

The shape of the data you collect constrains the questions you can ask later. Metrics
pre-aggregate context away at instrumentation time. Wide events preserve context and
let you decide the shape of your analysis at query time.

Every attribute on a span is a queryable dimension. Adding `user.id`, `deployment.version`,
and `cache.hit` to the same span lets you correlate them in a single query — "slow
requests are from tenant X on version 2.3.1 with cache misses." Separate metrics can't
do this because each dimension combination creates a new time series.

Honeycomb's storage engine handles high cardinality and dimensionality without the
cost explosion that affects metrics systems. Adding a high-cardinality field like
`user.id` doesn't create millions of time series — it's another column on each event,
aggregated at query time.

## Events vs Metrics vs Logs

| | Structured Events (Spans) | Metrics | Logs |
|---|---|---|---|
| **Captures** | Full request context (all attributes) | Pre-aggregated numbers with low-cardinality tags | Text or structured fields per line |
| **Discards** | Nothing — raw events retained | Individual requests, high-cardinality dimensions | Correlation across lines (without trace context) |
| **Query power** | GROUP BY, filter, BubbleUp on any dimension | Fast aggregates on pre-defined dimensions | Text search, structured field queries |
| **Cost scaling** | Linear with event volume | Exponential with dimension count (cardinality) | Linear with volume, query cost varies |
| **Best for** | Investigation, root cause analysis | Cheap alerting, long-term trends | Audit trails, rare events |

The same instrumentation effort that produces a metric or log line can produce a wide
event — 

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 1 — file references are metadata, not package contents.

otel-genai-instrumentation30.4 KB

View saved version →

---
name: otel-genai-instrumentation
description: >
  Guides instrumentation of GenAI/LLM applications with OpenTelemetry
  for Honeycomb, including content capture and agent failure detection.
  Trigger phrases: "instrument my GenAI app", "add tracing to LLM calls",
  "trace AI agent", "instrument OpenAI", "instrument Anthropic",
  "GenAI observability", "trace tool calling", "LLM token usage",
  "instrument embeddings", "trace MCP", "GenAI metrics",
  "instrument LangChain", "add GenAI spans", "capture prompts",
  "capture LLM responses", "enable GenAI content capture",
  "streaming tracing", "trace streaming responses",
  or any request about instrumenting GenAI/LLM applications.
metadata:
  version: "1.0.0"
  semconv_version: "v1.40.0"
---

# GenAI Instrumentation for Honeycomb

Instrumenting LLM and agent applications using OTel Semantic Conventions for GenAI
(currently v1.40.0, Development status). For conceptual foundations, see
the **observability-fundamentals** skill.

## Base OTEL Setup (Required First)

**BEFORE implementing GenAI instrumentation, ensure your base OpenTelemetry configuration is complete.**

Use the **otel-instrumentation** skill to configure all standard OTEL environment variables
(OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, OTEL_EXPORTER_OTLP_PROTOCOL,
signal-specific endpoints, etc.) and verify basic spans are flowing to Honeycomb.

GenAI instrumentation adds GenAI-specific configuration on top of that base setup.

## Critical Requirements (Non-Negotiable)

**BEFORE implementing any GenAI instrumentation, complete these steps in order:**

### Step 1: Ask About Content Capture (FIRST!)

**Stop and ask the user this question BEFORE writing any code or configuration:**

> "Do you want to capture the actual prompts and model responses in your traces?
>
> **Enabling content capture:**
> - ✅ Helps debug tool call failures, planning loops, and agent deadlocks
> - ✅ Lets you see why the model made specific decisions
> - ❌ Captures potentially sensitive content (user prompts, model responses)
> - ❌ May contain PII, proprietary data, or confidential information
>
> **Recommended for:** debugging/development, non-sensitive data, or if you have filtering
>
> **Not recommended for:** production with sensitive data, PII/health/financial info"

**Record their answer** — you'll need it when configuring instrumentation.

### Step 2: Enable GenAI Conventions (REQUIRED)

```bash
export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
```

Without this, GenAI spans will not be created.

### Step 3: Set Required Attributes on EVERY Span (REQUIRED)

- `gen_ai.operation.name` — e.g., `chat`, `execute_tool`, `invoke_agent`
- `gen_ai.conversation.id` — same value for all spans in a conversation

**Impact if missing**: Spans won't be recognized as GenAI operations and cannot be queried by session.

### Step 4: Implement force_flush() (REQUIRED)

GenAI apps often exit early (crash, Ctrl+C, CLI). Force flush after each top-level invocation
to prevent silent span loss.

For OTLP configuration, environment variables, and Honeycomb authentication (including the
silent-rejection pitfall), see the **otel-instrumentation** skill.

## Prerequisites

**This skill assumes your agent application is already sending telemetry to Honeycomb.** You should have:
- OpenTelemetry SDK installed and initialized
- All standard OTEL environment variables configured (see **Base OTEL Setup** section above)
- OTLP exporter configured with your Honeycomb API key
- Basic spans flowing to Honeycomb

**If you haven't set this up yet, use the otel-instrumentation skill first** for:
- SDK setup and dependencies
- OTEL environment variables (OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_*, etc.)
- OTLP configuration and Honeycomb authentication
- Verification that spans are flowing

Once base telemetry is working, return here to add GenAI-specific instrumentation.

## Auto-Instrumentation (Python and Node.js)

Python and Nod

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 7 — file references are metadata, not package contents.

otel-instrumentation18.1 KB

View saved version →

---
name: otel-instrumentation
description: >
  Provides guidance on OpenTelemetry SDK setup, custom instrumentation,
  and sending data to Honeycomb.
  Trigger phrases: "instrument my app", "add tracing",
  "set up OpenTelemetry", "configure OTel", "add custom spans",
  "add attributes to spans", "send traces to Honeycomb",
  "set up OTLP", "configure sampling", "add span events",
  "add span links", "set up tracing for [any language]",
  "configure the OTel Collector",
  or any request about OpenTelemetry SDK setup, custom instrumentation,
  or sending data to Honeycomb.
metadata:
  version: "1.0.0"
---

# OpenTelemetry Instrumentation for Honeycomb

SDK setup, custom spans, attributes, span events, sampling, and layered telemetry.
For conceptual foundations (why wide events matter, how attributes connect to
investigation), see the **observability-fundamentals** skill.

## OTLP Configuration and SDK Setup

Every OTel SDK needs these environment variables to send data to Honeycomb:

### Required Environment Variables

**Base configuration:**
```bash
OTEL_SERVICE_NAME=your-service-name
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.honeycomb.io
OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=YOUR_API_KEY"
```

**Optional but recommended:**
```bash
# Protocol selection (default: http/protobuf)
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf  # or grpc

# Signal-specific endpoints (override base endpoint for specific signals)
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.honeycomb.io/v1/traces
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=https://api.honeycomb.io/v1/metrics
```

**For metrics (preferred):** Use modern OTLP metrics and native datapoints. Use dataset
hints to confirm the destination type (`metrics` or `events`). Authenticate with:
```bash
OTEL_EXPORTER_OTLP_METRICS_HEADERS="x-honeycomb-team=YOUR_API_KEY"
```

### Protocol Selection

`OTEL_EXPORTER_OTLP_PROTOCOL` determines the wire format and transport:
- `http/protobuf` (default, recommended) — HTTP with protobuf encoding
- `grpc` — gRPC with protobuf encoding
- `http/json` — HTTP with JSON encoding (larger payload, slower)

Use `http/protobuf` unless you have specific infrastructure requirements for gRPC.

### Signal-Specific Endpoints

By default, OTel SDKs append `/v1/traces` and `/v1/metrics` to `OTEL_EXPORTER_OTLP_ENDPOINT`.
Use signal-specific endpoint vars to override:
- `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` — full URL for traces (including `/v1/traces`)
- `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` — full URL for metrics (including `/v1/metrics`)

Useful when routing signals to different backends or using non-standard endpoints.

### Common Pitfalls

**Silent auth failure:** The OTLP exporters need the `x-honeycomb-team` header to
authenticate. Without it, Honeycomb silently rejects requests — no error, no data. Set
`OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=YOUR_API_KEY"` or pass headers
programmatically. If loading the key from `.env`, ensure dotenv runs before SDK init.

**Metrics:** Prefer modern OTLP metrics and native datapoints. Dataset hints identify the
destination type (`metrics` or `events`), so do not add `x-honeycomb-dataset` by default.
Use that header only when hints or configuration require legacy routing to a named event
dataset. Traces do not need it; they route by `service.name`.

For the env var values, language-specific dependencies, and setup code (Go, Python,
Node.js, Java, Ruby, .NET, Rust), see
`${CLAUDE_PLUGIN_ROOT}/skills/otel-instrumentation/references/sdk-setup-by-language.md`.

## Custom Instrumentation

### Adding Attributes to Existing Spans (Highest Impact)

Add business context to auto-instrumented spans — no new spans needed. Get the current
span from context and call `SetAttributes` (Go), `set_attribute` (Python), or
`setAttribute` (Node.js) with user, tenant, business, and deployment context.

### Creating Custom Spans

Wrap important business operations for visibility in the trace waterfall. Use
`tracer.Start(ctx, "operation-name")` (Go), `tracer

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 8 — file references are metadata, not package contents.

otel-migration8.68 KB

View saved version →

---
name: otel-migration
description: >
  Guide for retrofitting OpenTelemetry into an existing, uninstrumented application.
  Trigger phrases: "migrate existing app to OTel",
  "add OpenTelemetry to existing project", "retrofit OTel into my codebase",
  "thread context through my code", "context propagation",
  "bridge Prometheus metrics to OTel", "logging bridge",
  "migrate logging to OTel", "slog bridge", "logback bridge",
  "verify my instrumentation", "traces are disconnected",
  "orphaned spans", "migrate to OpenTelemetry", "OTel migration plan",
  "how do I sequence an OTel migration", "add tracing to existing code",
  "refactor for context propagation", "Fiber context gotcha",
  "keep existing logging working with OTel", "add OTel without breaking Prometheus",
  "bridge existing metrics", "coexist with existing monitoring",
  or any request about retrofitting OpenTelemetry into an existing application.
  This skill is for migrating existing codebases, NOT greenfield instrumentation (use otel-instrumentation)
  or Beeline-specific migration (use beeline-migration).
metadata:
  version: "1.0.0"
---

# OpenTelemetry Migration for Existing Applications

Guide for retrofitting OpenTelemetry into an existing, uninstrumented application. This covers
the phased migration approach, context propagation refactoring, logging and metrics bridges, and
verification. This is distinct from greenfield OTel setup (see otel-instrumentation skill) and
Beeline-specific migration (see beeline-migration skill).

## When to Use This Skill

Use this skill when the user has an **existing application** that:
- Has no OpenTelemetry instrumentation and needs to add it
- Has existing logging, metrics, or context patterns that must coexist with OTel
- Needs to refactor function signatures to thread trace context through the call stack
- Uses a framework with OTel middleware gotchas (e.g., Fiber, Gin, Express)

For greenfield OTel setup, use the `otel-instrumentation` skill instead.
For Beeline-to-OTel migration, use the `beeline-migration` skill instead.
For understanding *why* to instrument, see the `observability-fundamentals` skill.

## Migration Phases

The migration follows six phases in order. Each phase is independently deployable and verifiable.
Context propagation (Phase 3) is typically ~60% of the effort.

### Phase 1: SDK Initialization and Shutdown

Set up TracerProvider, MeterProvider, and LoggerProvider with OTLP exporters. Wire initialization
early in the application's entry point and shutdown in signal handlers.

**Key guidance:**
- SDK init must happen *before* any application code that might create spans — one of the first
  things in your entry point, before config loading or storage initialization
- Shutdown ordering matters: flush traces, then metrics, then logs. Use a timeout (10-30s)
- If init fails, the application should still work — log the error and continue without telemetry
- For language-specific SDK setup, consult
  `${CLAUDE_PLUGIN_ROOT}/skills/otel-instrumentation/references/sdk-setup-by-language.md`

### Phase 2: HTTP Middleware (Auto-Instrumentation)

Add OTel middleware to your HTTP framework. This gives you automatic spans for every inbound
request with zero code changes to handlers. This is the highest-ROI step.

**Critical:** Different frameworks expose the OTel-enriched context differently. This is the #1
source of silent trace breaks. Consult
`${CLAUDE_PLUGIN_ROOT}/skills/otel-migration/references/framework-middleware.md` for
framework-specific details.

| Framework | How to get OTel context | Common mistake |
|-----------|------------------------|----------------|
| Go net/http | `r.Context()` | N/A (standard) |
| Go Fiber v2 | `c.UserContext()` | Using `c.Context()` (returns fasthttp context without OTel span) |
| Go Gin | `c.Request.Context()` | Using `c` directly |
| Go Echo | `c.Request().Context()` | N/A |
| Python Flask | Automatic (thread-local) | N/A with instrumentation library |
| Python Django | A

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 5 — file references are metadata, not package contents.

production-investigation7.65 KB

View saved version →

---
name: production-investigation
description: >
  Structured workflows for investigating production issues in Honeycomb — the
  sequence of tool calls (context priming, broad query, BubbleUp, trace analysis,
  verification) and how to chain results between steps to reach root causes.
  Trigger phrases: "investigate production issue", "debug latency spike",
  "find root cause", "use BubbleUp", "analyze traces", "debug an outage",
  "why is my API slow", "errors are increasing", "health check", "SLO burning",
  or any request to investigate or debug production problems.
metadata:
  version: "1.0.0"
---

# Honeycomb Production Investigation

Structured workflows for debugging production issues. The MCP tools document their
own parameters — this skill focuses on the *sequence* of tool calls and how to
*interpret* results to reach a root cause.

## The Core Analysis Loop

This workflow implements the core analysis loop (**Define → Visualize → Investigate →
Evaluate**) from the **observability-fundamentals** skill. If BubbleUp returns nothing
useful, the issue is often an instrumentation gap — add the missing attributes (see the
**otel-instrumentation** skill) and try again.

## Investigation Workflow

### Step 1: Orient
1. `get_workspace_context` → environments and datasets
2. `get_slos` → any SLOs in violation? (frames severity)
3. `get_triggers` → any alerts firing? (narrows scope)
4. `find_queries` → has anyone investigated this before?

### Step 2: Characterize the Problem
Run a broad query to see the shape of the issue:
- **Latency spike**: P99(duration_ms), HEATMAP(duration_ms) grouped by service or route
- **Error surge**: count failed operation spans (`error=true`) by service/route/category, then
  separately count exception event rows using `event.name=exception` and `exception.type exists`;
  use sampled `trace.trace_id` values to drill into representative traces
- **Unknown**: COUNT grouped by service.name to find which service has anomalous volume

Also call `get_service_map` — it shows P95 durations between services and can immediately reveal which dependency is slow.

**Exception data has two query surfaces:** operation failures belong on spans (`error=true`, span
status, low-cardinality `exception.slug`/error category); full exception diagnostics may belong on
trace-correlated Logs API event rows. Do not assume `exception.*` exists on the containing span.
When investigating exceptions, discover the dataset schema first, query `event.name=exception`
with `exception.type exists` and `trace.trace_id exists`, take a sample, then pass its
`trace.trace_id` to `get_trace(show_events=true)`. For legacy span-event exceptions, also check
`name=exception` and `meta.signal_type=trace`; Logs API events use `event.name`/`body` and
`meta.signal_type=log`.

If a service uses an exception-promoting `LogRecordProcessor`, some `exception.*` fields may also
appear on the containing span. Treat that as an explicit client-side compatibility feature, not a
Honeycomb guarantee: the event row remains authoritative for full diagnostics, and absence of
parent-span fields does not mean the exception event is missing.

### Step 3: BubbleUp to Find Differentiators
This is the highest-value step. Once you have a query showing the anomaly:
1. Run `run_bubbleup` on the query result, selecting the outlier region
2. BubbleUp compares outlier vs baseline distributions across *all* columns automatically
3. Look for fields where the distributions differ significantly

**How to interpret BubbleUp results:**
- **Categorical fields** (dimensions): A value overrepresented in outliers points to a cause (e.g., `deployment.version=v2.3.1` is 90% of slow requests but only 20% of baseline)
- **Numeric fields** (measures): A shifted distribution shows correlated metrics (e.g., `db.query_duration` is much higher in outliers)
- **Typical root causes surfaced**: deployment version, region, user cohort, specific endpoint, feature flag

### Step 4: Drill Into Traces

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 3 — file references are metadata, not package contents.

query-patterns7.11 KB

View saved version →

---
name: query-patterns
description: >
  Opinionated guidance for constructing and interpreting Honeycomb queries on
  trace and event datasets — operation selection (percentiles not AVG, HEATMAP
  for distributions), relational field patterns (root., parent., any., none.),
  calculated fields, query math, and result interpretation (P99/P50 ratios,
  heatmap bands, TOTAL/OTHER rows, raw JSON via query_result_json). Use this
  skill when the user wants to query spans, traces, or log/event data in
  Honeycomb — requests like "show me latency", "error rate", "find slow
  requests", "find outliers", "interpret results", "relational fields",
  "calculated fields", or "download raw results". This skill covers all
  dataset types except metrics datasets (dataset_type=metrics) — for those,
  use metrics-queries instead.
metadata:
  version: "1.1.0"
---

# Honeycomb Query Patterns

Opinionated guidance for writing effective Honeycomb queries. The MCP tools already
document their parameters and schemas — this skill focuses on *when* and *why* to
use each pattern, not *how* to call the tools.

## Key Principles

1. **Never use AVG for latency** — AVG hides tail latency. Use P99 (or P95/P90) to see what slow users experience. Reserve AVG for non-latency metrics like payload size.
2. **Use HEATMAP for distributions** — Single-number aggregates hide bimodal patterns. HEATMAP reveals whether you have one population or two.
3. **Combine calculations in one query** — `COUNT, P99(duration_ms), HEATMAP(duration_ms)` in a single query reduces API calls and gives a complete picture.
4. **Start broad, narrow with WHERE** — Begin with a COUNT/GROUP BY to understand shape, then add filters to focus.
5. **Check for prior work** — Call `find_queries` before writing new queries. Someone may have already answered the question.

## Choosing the Right Operation

| Question | Use |
|----------|-----|
| How much traffic? | `COUNT` grouped by route or service |
| How many unique users/IPs? | `COUNT_DISTINCT(field)` |
| How fast for most users? | `P50(duration_ms)` |
| How fast for the worst-off users? | `P99(duration_ms)` |
| Is there a bimodal pattern? | `HEATMAP(duration_ms)` |
| What's the worst case? | `MAX(duration_ms)` |
| How many concurrent operations? | `CONCURRENCY` |
| Is it getting worse over time? | `RATE_AVG(duration_ms)` |

## Relational Field Strategy

Use relational prefixes to ask cross-span questions within a trace:

- **"Show me slow endpoints caused by a specific downstream"**: Filter with `any.service.name` to find traces where that service participates, group by `root.http.route` to see which user-facing endpoints are affected.
- **"What's different about errored traces?"**: Filter with `any.error = true`, group by `root.name` to see which entry points have errors somewhere in their trace tree.
- **Exclude noise**: `none.service.name = "health-check"` removes traces containing health checks.

## Calculated Fields

Calculated fields are per-event expressions evaluated at query time. They transform,
classify, and combine existing fields without re-instrumenting code.

**Three scopes** — choose the narrowest that fits the need:
- **Query-scoped** (not saved): exploratory, one-off analysis
- **Dataset-level** (saved): reusable within one service's dataset
- **Environment-level** (saved): reusable across all datasets (e.g., `error_pct`)

**Common patterns:**
- **Error rate**: `MUL(IF($error, 1, 0), 100)` → use `AVG(error_pct)` to get percentage
- **Status classification**: `IF(GTE($http.status_code, 500), "5xx", GTE($http.status_code, 400), "4xx", "ok")`
- **Latency bucketing**: `BUCKET($duration_ms, 500, 0, 3000)`
- **Prefix routing**: `IF(STARTS_WITH($url, "/admin"), "admin", STARTS_WITH($url, "/api"), "api", "other")`
- **Exact-match classification**: use `SWITCH` instead of `IF(EQUALS(...))` chains — same expression, more efficient

**Key guardrails:**
- **Don't create presentational (alias-only) fields** — a field that just renames ano

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 5 — file references are metadata, not package contents.

slos-and-triggers6.4 KB

View saved version →

---
name: slos-and-triggers
description: >
  Decision heuristics for interpreting Honeycomb SLO compliance, budget burn rates,
  and trigger status — what the numbers mean and what action to take, including
  detecting misconfigured SLIs, deciding when to freeze deploys vs page on-call,
  and designing burn alert thresholds. Load this skill before calling get_slos or
  get_triggers.
  Trigger phrases: "check our SLOs", "are we meeting our SLOs", "which SLOs are
  healthy", "is the error budget OK", "are any alerts firing", "what's the burn rate",
  "set up an SLO", "create a trigger", "configure alerts", "set up burn alerts",
  "check trigger status", "starting on-call", "reliability picture",
  "should we freeze deploys", "is this SLO misconfigured", "are we within budget",
  "SLO is broken", "budget is negative", or any request about service level
  objectives, error budgets, burn rates, or alerting in Honeycomb.
metadata:
  version: "1.0.0"
---

# Honeycomb SLOs and Triggers

Guidance for configuring and reasoning about reliability in Honeycomb. The `get_slos`
and `get_triggers` tools document their own parameters — this skill focuses on
_designing_ effective SLOs, _choosing_ between SLOs and triggers, and _interpreting_
what the numbers mean.

**Availability**: SLOs require Pro or Enterprise plan. Triggers available on all plans.

## SLO vs Trigger — When to Use Which

| Question                                      | SLO                    | Trigger |
| --------------------------------------------- | ---------------------- | ------- |
| "Are we meeting our reliability commitments?" | Yes                    | No      |
| "Is something broken right now?"              | No                     | Yes     |
| "How fast are we burning our error budget?"   | Yes (burn alerts)      | No      |
| "Did error count exceed a threshold?"         | No                     | Yes     |
| "Should we slow down deploys?"                | Yes (budget remaining) | No      |

**Rule of thumb**: SLOs measure reliability against commitments over time. Triggers catch immediate operational issues.

## Designing Effective SLOs

### Define the SLI

An SLI is a per-event boolean: was this event successful? Implemented as a calculated field returning undefined (not a relevant event), 1 (success), or 0 (failure).

- **Format**: `IF(<qualifying-condition>, <success-condition>)` The qualifying condition filters to relevant events; the success condition defines what counts as success. If the qualifying condition is not met, the formula returns undefined, and the SLI is unpopulated.
- **Specific Qualifying Condition**: Choose the relevant subset of events (e.g. `AND(EQUALS($http.route, "/checkout"), NOT(EXISTS($trace.parent_id)))` for root spans of checkout endpoint)
- **Latency Success Condition**: `LTE(duration_ms, 500)` — requests faster than 500ms
- **Availability Success Condition**: `LTE(http.status_code, 499)` — non-5xx responses
- **Business Logic Success Condition**: `EQUALS(checkout.status, "completed")` — successful checkouts

### Set the Target

- Start conservative (99% before 99.99%)
- Measure current baseline first with P50/P99 queries
- Set target slightly above current performance
- Ask: what reliability do users actually need?

### Configure Exhaustion Time Alerts

At minimum, two alerts:

- **Near exhaustion** (exhaustion time ~4h): pages on-call via PagerDuty
- **Trending to exhaustion** (budget rate over 24h): notifies team via Slack

### Configure Burn Rate Alerts

Detect fast burns even if the budget isn't close to exhaustion yet. For example:

- 1h burn rate > 10x — page on-call

Recommend these alerts to the user after creating the SLO. Agents do not have the ability to set up these alerts or their recipients.

### Best Practices

- Measure close to the user (at the edge, not deep in the stack)
- Design around user workflows, not team boundaries
- Favor broad SLOs over many narrow ones
- Start with one SLO, reduce noise, then expand

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 3 — file references are metadata, not package contents.

verify-recent-trace2.9 KB

View saved version →

---
name: verify-recent-trace
description: >
  Find a recent trace in Honeycomb, to see what happened in a recent test. Use when asked to "verify in Honeycomb", "find the most recent trace", or "show me the trace".
metadata:
  version: "1.0.0"
allowed-tools:
  - mcp__honeycomb__run_query
  - mcp__honeycomb__get_trace
  - mcp__honeycomb__get_query_results
  - mcp__honeycomb__find_queries
  - Read
---

# Honeycomb Verification Skill

## Purpose

Query Honeycomb to find the traces that were created by a recent test.

## When to Use

This skill is automatically activated when:

- User asks to "show me a trace in Honeycomb"
- Completing an implementation step that requires observability verification

## Verification Output

Report query results as:
Queried [VIZ] where [FILTERS] by [GROUP_BY] over [TIME_RANGE]
Results: [link to Honeycomb query]

## Query Patterns

- **Time Range**: 1200 seconds (20 minutes) - adjustable based on when the test started

### Pattern 1: Get a particular trace

What service are you testing? This determines the dataset.
Do you know the name of the span you expect to see?

```
Query for:
- dataset: [dataset]
- filter: name = [expected span name]
- time_range: [calculated time range]
- calculation: COUNT
- include_samples: true
```

Now take the most recent sample, and use get_trace to get the full trace.
Output:

Queried [CALCULATION] where [FILTERS] over [TIME_RANGE]
Found [count] results
Results: [link to the query]

Most recent trace: [link to the trace]
Root span: [root span name]
Total spans: [count of spans]
Services: [list of services]

### Pattern 2: Find all traces since the test started

```
Query for:
- all datasets
- filter: trace.parent_id does-not-exist
- time_range: [calculated time range]
- calculation: COUNT
- breakdowns: name
- include_samples: true
```

Output:
Queried [CALCULATION] where [FILTERS] by [BREAKDOWNS] over [TIME_RANGE]
Found [count] results
Results: [link to the query]

For each sample, print:

- Trace ID
- Span name
- anything else it gives you
  Construct a link to the trace in Honeycomb, according to:
  https://ui.honeycomb.io/modernity/environments/personal-agent/trace?trace_id=[<traceId>]
  &trace_start_ts=[test_start_time]
  &trace_end_ts=[((date +%s))]

### Pattern 3: Find the most recent trace, precisely

```
Query for:
  calculated_fields: event_time=EVENT_TIMESTAMP()
  calculation: COUNT
  breakdowns: event_time, trace.trace_id
  order: event_time DESC
  limit: 1
  include_samples: false
```

Print the summary of the trace, as in Pattern 1.

Take the output trace_id and use get_trace to get the full trace.

### Pattern 4: When you don't see any traces, try: Get the time range right

Before running a test that will generate a trace, print the current time.

start_time=$(date +%s)

Then run the test to create the trace.

Then calculate the correct time range:

echo $(( $(date +%s) - $start_time + 20))

Include that in the query you're using to find traces.
Package details

Publisher declarations from the archived package. These are separate from our research and the live service's terms.

Package author
honeycomb.io

Package observed Nov 21, 2026.

Technical details
First seen
Sep 30, 2026 · 22:02 UTC
Last seen
Nov 22, 2026 · 00:00 UTC
Latest observed change
Oct 6, 2026 · 18:04 UTC
Collection status
Collected

plugin_asdk_app_6a51b29fa2d48191b215ff28f9a64fb4

Download plugin data (JSON)

Plugin details: freshly prepared. Prepared . Refreshed when plugin data changes or after 6 hours. Your account and Follow status are live.