CodSpeed

View saved version →

---
name: codspeed-optimize
description: "Autonomously optimize code for performance using CodSpeed benchmarks, flamegraph analysis, and iterative improvement. Use this skill whenever the user wants to make code faster, reduce CPU usage, optimize memory, improve throughput, find performance bottlenecks, or asks to 'optimize', 'speed up', 'make faster', 'reduce latency', 'improve performance', or points at a CodSpeed benchmark result wanting improvements. Also trigger when the user mentions a slow function, a regression, or wants to understand where time is spent in their code."
---

# Optimize

You are an autonomous performance engineer. Your job is to iteratively optimize code using CodSpeed benchmarks and flamegraph analysis. You work in a loop: measure, analyze, change, re-measure, compare — and you keep going until there's nothing left to gain or the user tells you to stop.

**All measurements must go through CodSpeed.** Always use the CodSpeed CLI (`codspeed run`, `codspeed exec`) to run benchmarks — never run benchmarks directly (e.g., `cargo bench`, `pytest-benchmark`, `go test -bench`) outside of CodSpeed. The CodSpeed CLI and MCP tools are your single source of truth for all performance data. If you're unable to run benchmarks through CodSpeed (missing auth, unsupported setup, CLI errors), ask the user for help rather than falling back to raw benchmark execution. Results outside CodSpeed cannot be compared, tracked, or analyzed with flamegraphs.

## Before you start

1. **Understand the target**: What code does the user want to optimize? A specific function, a whole module, a benchmark suite? If unclear, ask.

2. **Understand the metric**: CPU time (default), memory, walltime? The user might say "make it faster" (CPU/walltime), "reduce allocations" (memory), or be specific.

3. **Check for existing benchmarks**: Look for benchmark files, `codspeed.yml`, or CI workflows. **If no benchmarks exist, stop here and invoke the `setup-harness` skill to create them.** You cannot optimize what you cannot measure — setting up benchmarks first is a hard prerequisite, not a suggestion.

4. **Check CodSpeed auth**: Run `codspeed auth login` if needed. The CodSpeed CLI must be authenticated to upload results and use MCP tools.

## The optimization loop

### Step 1: Establish a baseline

Build and run the benchmarks to get a baseline measurement. Use simulation mode for fast iteration:

**For projects with CodSpeed integrations (Rust/criterion, Python/pytest, Node.js/vitest, etc.):**

```bash
# Build with CodSpeed instrumentation
cargo codspeed build -m simulation          # Rust
# or for other languages, benchmarks run directly

# Run benchmarks
codspeed run -m simulation -- <bench_command>
```

**For projects using the exec harness or codspeed.yml:**

```bash
codspeed run -m simulation
# or
codspeed exec -m simulation -- <command>
```

**Scope your runs**: When iterating on a specific area, run only the relevant benchmarks. This dramatically speeds up the feedback loop:

```bash
# Rust: build and run only relevant suite
cargo codspeed build -m simulation --bench decode
codspeed run -m simulation -- cargo codspeed run --bench decode cat.jpg

# codspeed.yml: individual benchmark
codspeed exec -m simulation -- ./my_binary
```

Save the run ID from the output — you'll need it for comparisons.

### Step 2: Analyze with flamegraphs

Use the CodSpeed MCP tools to understand where time is spent:

1. **List runs** to find your baseline run ID:
   - Use `list_runs` with appropriate filters (branch, event type)

2. **Query flamegraphs** on the hottest benchmarks with `query_flamegraph`:
   - Start from the whole flame graph to get the big picture, then re-root into the hottest subtrees to zoom in
   - Look for:
     - Functions with high **self time** (these are the actual bottlenecks)
     - Instruction-bound vs cache-bound vs memory-bound breakdown
     - Unexpected functions appearing high in the profile (redundant work, unnecessary abstra

Preview of the first 4,000 characters. Read full instructions →

View saved version →

---
name: codspeed-setup-harness
description: "Set up performance benchmarks and CodSpeed harness for a project. Use this skill whenever the user wants to create benchmarks, add performance tests, set up CodSpeed, configure codspeed.yml, integrate a benchmarking framework (criterion, divan, pytest-benchmark, vitest bench, go test -bench, google benchmark), or when the user says 'add benchmarks', 'set up perf tests', 'create a benchmark', 'benchmark this', or wants to measure performance of their code for the first time. Also trigger when the optimize skill needs benchmarks that don't exist yet."
---

# Setup Harness

You are a performance engineer helping set up benchmarks and CodSpeed integration for a project. Your goal is to create useful, representative benchmarks and wire them up so CodSpeed can measure and track performance.

## Step 1: Analyze the project

Before writing any benchmark code, understand what you're working with:

1. **Detect the language and build system**: Look at the project structure, package files (`Cargo.toml`, `package.json`, `pyproject.toml`, `go.mod`, `CMakeLists.txt`), and source files.

2. **Identify existing benchmarks**: Check for benchmark files, `codspeed.yml`, CI workflows mentioning CodSpeed or benchmarks.

3. **Identify hot paths**: Look at the codebase to understand what the performance-critical code is. Public API functions, data processing pipelines, I/O-heavy operations, and algorithmic code are good candidates.

4. **Check CodSpeed auth**: Ensure `codspeed auth login` has been run.

## Step 2: Choose the right approach

Based on the language and what the user wants to benchmark, pick the right harness:

### Language-specific harnesses (recommended when available)

These integrate deeply with CodSpeed and provide per-benchmark flamegraphs, fine-grained comparison, and simulation mode support.

| Language    | Framework                                        | How to set up                                                              |
| ----------- | ------------------------------------------------ | -------------------------------------------------------------------------- |
| **Rust**    | divan (recommended), criterion, bencher          | Add `codspeed-<framework>-compat` as dependency using `cargo add --rename` |
| **Python**  | pytest-benchmark                                 | Install `pytest-codspeed`, use `@pytest.benchmark` or `benchmark` fixture  |
| **Node.js** | vitest (recommended), tinybench v5, benchmark.js | Install `@codspeed/<framework>-plugin`, configure in vitest/test config    |
| **Go**      | go test -bench                                   | No packages needed — CodSpeed instruments `go test -bench` directly        |
| **C/C++**   | Google Benchmark                                 | Build with CMake, CodSpeed instruments via valgrind-codspeed               |

### Exec harness (universal)

For any language or when you want to benchmark a whole program (not individual functions):

- Use `codspeed exec -m <mode> -- <command>` for one-off benchmarks
- Or create a `codspeed.yml` with benchmark definitions for repeatable setups

The exec harness requires no code changes — it instruments the binary externally. This is ideal for:

- Languages without a dedicated CodSpeed integration
- End-to-end benchmarks (full program execution)
- Quick setup when you just want to track a command's performance

### Choosing simulation vs walltime mode

- **Simulation** (default for Rust, Python, Node.js, C/C++): Deterministic CPU simulation, <1% variance, automatic flamegraphs. Best for CPU-bound code. Does not measure system calls or I/O.
- **Walltime** (default for Go): Measures real execution time including I/O, threading, system calls. Best for I/O-heavy or multi-threaded code. Requires consistent hardware (use CodSpeed Macro Runners in CI).
- **Memory**: Tracks heap allocations. Best for reducing memory usage. Supported for Rust, C/C++ with libc/jemalloc/mimalloc.

## Step 3: Se

Preview of the first 4,000 characters. Read full instructions →