AMD

View saved version →

---
name: hyperloom-workload-optimizer
description: >-
  Autonomously optimizes end-to-end LLM inference throughput on AMD Instinct GPUs
  and reports a validated gain, using the Hyperloom multi-agent optimizer. Given a
  model, framework, workload (TP/EP, concurrency, ISL/OSL, precision), an objective
  and a time budget, it explores per-workload which levers to pull (serving/config
  parameters and env, framework enablement and source patches, and hot GPU-kernel
  rewrites), benchmarks each candidate, and returns the optimization stack that
  produced the gain. Use when the user wants to make a model serve faster, raise
  tokens/sec or throughput, optimize or tune vLLM or SGLang on MI300X/MI325X/MI355X,
  run Hyperloom, run the kernel-agent, quantize-then-optimize with Quark, set up
  Hyperloom from scratch, or resume a Hyperloom session. Do not use to stand up a
  server for plain serving, diagnose a broken ROCm install, or run a one-off
  kernel/benchmark or trace analysis without the optimization loop.
---

<!--
Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved.

See LICENSE for license information.
-->

# Hyperloom Workload Optimizer

You are the entry point for Hyperloom optimization on AMD Instinct GPUs. Your job
is the bootstrap: confirm the workspace, install the Hyperloom wheel, run
`/hyperloom-setup`, then hand the run to the skill that owns it.

The wheel installs the skills that own everything after setup: `hyperloom-setup`
for credentials and run mode, the demo skills for a workload preset, and
`inference_optimizer` for the launcher gates and monitoring. They ship with the
runtime, so they always match the installed version.

## Out of scope for this skill

- Do not run `python -m hyperloom.inference_optimizer.cli optimize` yourself.
- Do not implement a GPU preflight, launcher gate, or background launch here. The
  installed skills own those, including the Iron Rules and the resume path.
- Do not ask for workload values (model, TP/EP, concurrency, ISL/OSL, precision,
  objective, budget) while installing or while setup is running. They belong to
  the run skill, after setup finishes.
- Do not optimize by hand in chat.

## Prerequisites

- AMD Instinct GPU host (MI300X / MI325X / MI355X) with ROCm, `/dev/kfd` and
  `/dev/dri` present, and `amd-smi` or `rocm-smi` working.
- Python 3.10+ and `pip` on the machine that runs the install.
- LLM credentials: Anthropic API access, or the AMD LLM gateway.
- A dedicated empty directory, opened in the agent as the workspace.

Confirm the shell you are in is on the GPU host before installing. Setup may later
point Docker at a different target host; until it does, everything here runs where
the agent is.

## Step 1: Confirm the workspace

The current directory is both the install target and the agent workspace. Confirm
with the user that it is a dedicated directory before installing: setup creates or
updates `.env` in it. Do not switch to another directory on your own, and do not
install into an existing project unless the user accepts the `.env` change.

## Step 2: Install the Hyperloom wheel

```bash
pip install hyperloom-inference-optimizer --target .
```

Install the current release unless the user asks for a specific version. It is
normal for the directory to hold many Python package folders afterwards; the user
does not need to inspect them.

## Step 3: Run `/hyperloom-setup`

The wheel installs `hyperloom-setup` into the agent's skill directories
(`.agents/skills/`, `.claude/skills/`, `.cursor/skills/`). Run it:

```text
/hyperloom-setup
```

It is interactive and owns credentials, `USER_DATA_PATH`, the run mode
(`docker` recommended, or `baremetal`), the Docker target host, and the bare-metal
framework install. It writes `.env` and stops before any optimization. Run it once
per workspace; the run skills reuse those values.

Let setup ask its own questions. Do not preempt them, do not restate its option
lists, and never ask the user to paste an API k

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 4 — file references are metadata, not package contents.

View saved version →

---
name: local-ai-app-integration
description: >-
  Integrates local AI capabilities into applications using Embeddable Lemonade.
  Use when the user wants to add local AI, offline AI, private AI, on-device AI,
  a local LLM, local chat, embeddings, image generation, speech-to-text, or
  text-to-speech to an existing app; replace or supplement OpenAI, Anthropic, Ollama, or
  other cloud AI APIs with a local backend; only use to convert user apps. Do not use when
  the user just wants the agent itself to generate images, transcribe, or speak locally in
  the current workspace, even to cut their own API bill.
---

# Local AI App Integration (Embeddable Lemonade)

Add a local AI mode to an existing app that already talks to a cloud AI API
(OpenAI, Anthropic, or Ollama-compatible). The app launches `lemond`, the
Embeddable Lemonade binary, as a private subprocess and the existing client
talks to it on `http://localhost:PORT/api/v1`. The user gets local, private,
hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate
install.

**What you'll end up with:** one new launcher module (~30 lines), three mandatory changes to the existing HTTP client (`base_url`, `api_key`, and a 120-second HTTP timeout), one vendored binary under `vendor/lemonade/`.

## When this skill is the right tool

Use this skill when **all** of the following are true:

- The app already calls a cloud AI service over HTTP (OpenAI Chat Completions,
  Anthropic Messages, or Ollama).
- The user wants that AI to run on the end-user's PC, with the AI engine
  bundled into the app, not as a separate user install.
- The target platform is Windows x64 or Linux x64 (macOS embeddable is in beta).

If the user instead wants a **system-wide** Lemonade Server (one install,
shared across apps), do not use this skill; point them at
`https://lemonade-server.ai/install_options.html` and the standard OpenAI base
URL `http://localhost:13305/api/v1`.

## The opinionated path

This skill follows one fixed sequence. Do not deviate without a stated reason.

```
[ ] 1. Survey the app's current AI integration
[ ] 2. Pick a model + backend profile
[ ] 3. Place Embeddable Lemonade in the app's tree (full package, not just the binary)
[ ] 4. Add a `lemond` launcher (subprocess + API key + port + per-stage logging)
[ ] 5. Re-point the existing client at lemond (base_url, api_key, 120s timeout — all three required)
[ ] 6. Wait for /api/v1/health, install backend, then PULL the model before first use
[ ] 7. Wire shutdown and error recovery
```

Track progress against this checklist. Move on only when each step verifies.

> **Log every stage.** A local integration has many silent failure points —
> spawn, health, backend install, model download, first inference. Without a
> log line at each transition, "nothing happened" is indistinguishable from
> "broke at stage 3." Emit one clear line per stage as you build (see
> [Step 4](#step-4-add-a-lemond-launcher)); the most common dead-end in this
> integration — a blank result with no error — is invisible without them.

---

## Step 1: Survey the app

Find every place the app currently calls a cloud AI API. Search the repo for:

- `openai`, `OpenAI(`, `chat.completions`, `responses.create`
- `anthropic`, `Anthropic(`, `messages.create`
- `api.openai.com`, `api.anthropic.com`, `localhost:11434` (Ollama)
- `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`

Record three things before continuing:

1. **Client library and language** (e.g., `openai-python`, `openai-node`,
   `@anthropic-ai/sdk`, `go-openai`, raw `fetch`).
2. **Modalities used:** text chat, tool calling, embeddings, image gen,
   transcription, TTS. This drives the model + backend choice in Step 2.
3. **One single place** where the base URL and API key are constructed. If
   there isn't one, refactor to one before going further. Local-mode toggling
   must flip exactly one config object.
4. **Any API-key gating** that blocks the app before a key is entered
   (onboarding walls, val

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 6 — file references are metadata, not package contents.

View saved version →

---
name: local-ai-use
description: >-
  Makes this agent generate images, transcribe audio, and synthesize speech on
  the user's own machine through a local Lemonade Server instead of a paid cloud
  API. Use it above all to change that routing persistently, from now on — keep
  generating pictures locally while chat stays on the cloud; set this workspace
  up to make images on my own machine — even when the user asks for no image or
  file in the same breath. Also use it for a single request the user wants done
  locally, offline, on-device, or kept private: transcribe this recording, make
  this picture, read this text aloud. Applies in Claude, Cursor, Codex, or any
  agent harness. Use when the user wants to cut cost or tokens on image, audio,
  or voice API calls, or to drop DALL-E, hosted Whisper, ElevenLabs, or other
  paid multimodal APIs; or mentions Lemonade Server, OmniRouter, SD-Turbo,
  kokoro, Ryzen AI, or NPU/iGPU/dGPU inference. Changes no application source
  code; do not use it if the user is adding local AI to an app they ship.
---

# Local AI Use (route image, TTS, STT through Lemonade)

This is a **meta-skill**. You run it once. After that, every later request that
needs image generation, text-to-speech, or speech-to-text uses the local
[Lemonade Server](https://lemonade-server.ai) instead of a cloud API. The
agent's own LLM keeps handling text; only the expensive multimodal calls move
on-device.

The skill does three things:

1. **Makes sure local Lemonade is installed and running.** If no modern
   `lemonade` CLI is found, the setup script installs the latest version of
   Lemonade on the user's behalf. Modern Lemonade has no `serve` command — the
   Lemonade service (the `lemond` daemon) auto-starts on install and is managed
   by the OS — so the setup script waits for the service and, if it stays down,
   prints the exact OS-specific command to start it (e.g. `sudo systemctl start
   lemond` on Linux).
2. **Verifies that local Lemonade is reachable** on the port `lemonade status`
   reports, so a server already running on a non-default port is used as-is
   rather than reported missing (`13305` is only the fallback).
3. **Drops a `Local AI Use` block into the workspace `AGENTS.md`** so the agent
   reads the routing rule on every later turn, in Cursor, Claude Code, Codex,
   Gemini CLI, and any other agent that respects `AGENTS.md`.

> **Requires modern Lemonade (v10.1.0 or newer).** Modern Lemonade unified
> everything under one `lemonade` CLI (`lemonade status`, `lemonade pull`, ...)
> driving an always-on `lemond` service. `lemonade` is the only valid CLI, and
> this skill installs Lemonade only from the [official install
> paths](https://lemonade-server.ai/docs/guide/install/) listed in Step 1a. If
> an older `lemonade` is already on the `PATH` it will shadow the modern CLI;
> uninstall it first (see the removal commands in Step 1a) before running this
> skill.

Models are **not** downloaded during setup. Each default model is pulled
lazily, on first use, by the routing rule (e.g. the first image request pulls
the image model). This keeps setup fast and avoids gigabytes of downloads the
user may never need.

## When to use this skill

Use this skill when **all** of the following are true:

- The user wants local Lemonade. If it is not yet installed, the setup script
  installs the latest version for them automatically.
- The user accepts the Lemonade endpoint on this machine, whichever port the
  service reports (`http://localhost:13305` if it has never been changed).
- The user wants the change to be **persistent** across future turns and
  agent restarts (the rule is written to disk).

If the user is instead **embedding** Lemonade as a private subprocess inside
an app installer, do not use this skill; use `local-ai-app-integration`
instead.

## Prerequisites

- **OS:** Windows 11 x64, Ubuntu/Debian x64, or macOS (beta).
- **Lemonade:** the setup script installs the latest version if missing, usin

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 6 — file references are metadata, not package contents.

View saved version →

---
name: serving-llms-on-instinct
description: >-
  Serves AI models on AMD Instinct GPU hardware using vLLM. Use this skill
  whenever the user wants to run, serve, deploy, start, host, or launch a
  language model on an AMD GPU, AMD Instinct, MI300X, MI325X, MI350X, or MI355X.
  Also use when the user mentions vLLM on ROCm, vLLM on AMD, serving on HBM,
  or asks how to get a model running on AMD data center hardware. Use when the
  user asks "run Qwen3", "serve DeepSeek", "start a vLLM endpoint", "get a
  model running on my AMD machine", or any similar phrasing. Handles the full
  flow: GPU detection, environment validation, vLLM configuration, launch, and
  health verification. Do not use for NVIDIA GPUs, consumer AMD GPUs (RX
  series, Radeon), Ryzen AI, NPU, MI250X, or MI100.
allowed-tools: Bash, Read
---

# Serving LLMs on AMD Instinct

Get a vLLM endpoint running on AMD Instinct GPU hardware.

## Prerequisites

- ROCm driver and `amd-smi` installed on the GPU host
- Docker running and accessible (check with `docker ps`)
- `/dev/kfd` and `/dev/dri` present on the GPU host
- HuggingFace token in `HF_TOKEN` env var (required for gated models; not
  required for Qwen3 or Gemma). For gated models (Llama 3.2, Gemma, etc.),
  the HF token must belong to an account that has accepted the model's license
  at `huggingface.co/<model_id>`. A valid token without license acceptance will
  fail with an opaque "Engine core initialization failed" error.
- For remote GPU: SSH key access configured (`ssh <user>@<host>` must work
  without a password prompt). If only password access is available, set up
  keys first: `ssh-copy-id <user>@<host>`

## Data files

Read these files directly to get model and GPU configuration:

- **`data/recipes_cache.json`** -- model configs synced from
  [vllm-project/recipes](https://github.com/vllm-project/recipes). Each entry
  under `models.<HF_ID>.recipe` contains the full recipe with `model.base_args`,
  `model.base_env`, `features.tool_calling.args`, `features.reasoning.args`,
  `hardware_overrides.amd.extra_args`, `hardware_overrides.amd.extra_env`.
  The top-level `docker_image` field has the latest resolved vLLM ROCm image.

- **`data/gpu_overrides.json`** -- GPU-specific configuration. Contains
  `docker_flags` (mandatory for all AMD Instinct), `gpu_configs` keyed by
  gfx_version with `env_defaults` and `workarounds`, and `legacy_models` for
  models not yet in vLLM recipes.

- **`data/blacklist.json`** -- models in vLLM recipes that cannot be served
  as LLM endpoints. Includes diffusion/image/audio generation models, embedding
  models, rerankers, ASR models needing audio pipelines, and models requiring
  unreleased vLLM nightly builds. Check this before attempting to serve a model.
  If the user requests a blacklisted model, explain why it won't work and
  suggest an alternative.

If the user doesn't specify a model, default to **Qwen/Qwen3.5-9B**: dense
multimodal with MTP, Apache 2.0 license (no HF token needed), fits on a single
GPU, strong reasoning and tool-calling.

## Step 1: Detect the GPU

```bash
python3 scripts/detect.py
# Remote:
python3 scripts/detect.py --host user@hostname
```

Returns JSON with `gfx_version`, `vram_gb`, `gpu_count`, `rocm_version`.

| gfx_version | Hardware | VRAM |
|---|---|---|
| gfx950 | MI350X / MI355X | 288 GB HBM3E |
| gfx942 | MI300X (192 GB) / MI325X (256 GB) / MI300A (128 GB) | varies |

If `gfx_version` is `unknown`: `amd-smi` ran but found no GPU. Check
`lsmod | grep amdgpu`.

## Step 2: Validate the environment

```bash
python3 scripts/validate.py --auto-fix
# Remote:
python3 scripts/validate.py --auto-fix --host user@hostname
```

Returns JSON with `ready` (bool), `errors`, `warnings`, `fixes_applied`.
Do not proceed if `ready` is `false`.

## Step 3: Refresh recipes (if stale)

Check `fetched_at` in `data/recipes_cache.json`. If older than 24 hours or
the file is missing, refresh:

```bash
python3 scripts/sync_recipes.py
```

This shallow-clones v

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 12 — file references are metadata, not package contents.

View saved version →

---
name: tracelens-analysis-orchestrator
description: >-
  Orchestrates modular PyTorch profiler trace analysis with TraceLens: generates perf
  reports, prepares category data, runs system-level and compute-kernel subagents in
  parallel, validates outputs, and writes a prioritized stakeholder report (analysis.md).
  Use when the user asks to follow the analysis orchestrator, run the agentic analysis
  workflow, run it from config <run.yaml>, analyze a trace, compare two traces, or
  mentions standalone or comparative TraceLens analysis.
license: MIT
---

<!--
Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved.

See LICENSE for license information.
-->

# Analysis orchestrator

Coordinate **system-level** analysis (CPU/idle, kernel fusion, multi-kernel / comm / memcpy) and **compute-kernel** analysis (GEMM, SDPA, elementwise, etc.): one trace load, shared prep, parallel subagents, then aggregation into `analysis.md`.

## Full procedure

Follow **[reference.md](reference.md)** for every step (user prompts, `<prefix>` / `{CMD}` usage, CLI commands, subagent launch text, validation, report `tee` order, and plot embedding).

## Workflow index

```
0. Query User Inputs (interactive answers or a run YAML, Environment Setup) → cache/run_config.yaml
1. Generate Performance Report (one wrapper call on cache/run_config.yaml; semantic pre-processing first)
2. Trace-Quality Gate → Deterministic fallback analysis
3-6. Prepare Category Data (GPU Util, Top Ops, Tree Data, Multi-Kernel Data, Category Filtering)
7. System-Level Analysis (PARALLEL) → system_findings/
8. Compute Kernel Subagents (PARALLEL) → category_findings/
  8.5. Aggregate → priority_data.json::findings[]
9. Validate Subagent Outputs
10. load_findings + Model Identification (subagent) → metadata/model_info.json
    10.2. Analytical Roofline Augmentation (optional; only when <augment_analytical_roofline> is set)
11. Render performance PNG if agent_extension.py is absent
12. Generate analysis.md (orchestrator writes via <prefix> tee), optional extension, embed PNG
```

## Rules

- **Subagents:** Use your runner's subagent primitive (the Task tool in Claude Code or Cursor, `spawn_agent` under Codex) **only** where reference.md says “subagent” (Steps **7**, **8**, **10**, and **10.2**). The orchestrator runs everything else, including Step 8.5, using the command prefix from `<output_dir>/cache/cmd_prefix.txt` (`{CMD}` substitution).
- **Language:** Prefer vendor-agnostic terms (GPU kernels, collective communication, vendor GEMM library, DNN primitives, GPU graph). When quoting trace data, real kernel names are fine.
- **Subagent prompts:** Point each subagent at the checked-in agent file under `TraceLens/Agent/Analysis/skills/analysis-orchestrator/agents/<name>.md` (see reference.md for exact paths and prompt shells).

## Primary outputs

- **Deliverable:** `<output_dir>/analysis.md`
- **Internals:** `system_findings/`, `category_findings/`, `category_data/`, `metadata/`, `perf_report*.xlsx`, CSV folders — see package README for layout.

## Agent layout

Project subagents ship with this skill: `TraceLens/Agent/Analysis/skills/analysis-orchestrator/agents/*.md`.

Referenced files: 55 — file references are metadata, not package contents.