{"id":13766,"plugin_id":"plugin_asdk_app_6a971bba55a88191a9d4ad4edd5874b7","kind":"skill","collection_source":null,"comparison_source":null,"observed_at":"2026-09-30T23:08:06.744Z","digest":"afa0d0782427499302fc126072c0b84ff06d1e325dc948a92a17387280a3349f","against":null,"payload":{"description":"Use when integrating, instrumenting, evaluating, or testing an app with Openlayer — the AI evaluation and observability platform. Covers BOTH modes: (1) MONITORING/live — add tracing to LLM or agent code with the openlayer Python or TypeScript SDK (@trace, trace_openai / trace_anthropic / trace_litellm / trace_bedrock and the LangChain callback), publish inference rows, enrich traces with user/session context, metadata, and guardrails; and (2) DEVELOPMENT/offline — author openlayer.json + tests.json, push commits with the Openlayer CLI or MCP, gate CI/CD, and run the fix-loop on failing rows. Also covers creating tests (the prebuilt catalog, thresholds), inference pipelines, and programmatic data access. Trigger when the user mentions Openlayer, OPENLAYER_API_KEY, OPENLAYER_INFERENCE_PIPELINE_ID, openlayer.json, tests.json, an inference pipeline, \"openlayer push\", openlayer-mcp, trace_openai / trace_anthropic, Openlayer guardrails, or LLM evals.","included_files":[{"relative_path":"references/ci-cd.md","size_in_bytes":2654},{"relative_path":"references/cli.md","size_in_bytes":6314},{"relative_path":"references/custom-metrics.md","size_in_bytes":6043},{"relative_path":"references/data-access.md","size_in_bytes":2991},{"relative_path":"references/data-streaming.md","size_in_bytes":3036},{"relative_path":"references/development-setup.md","size_in_bytes":9038},{"relative_path":"references/docs-access.md","size_in_bytes":2049},{"relative_path":"references/gateway.md","size_in_bytes":3597},{"relative_path":"references/governance.md","size_in_bytes":7479},{"relative_path":"references/guardrails.md","size_in_bytes":6422},{"relative_path":"references/monitoring-instrumentation.md","size_in_bytes":10939},{"relative_path":"references/sdk-upgrade.md","size_in_bytes":1709},{"relative_path":"references/tests.md","size_in_bytes":8289},{"relative_path":"references/traditional-ml.md","size_in_bytes":5804},{"relative_path":"references/troubleshooting.md","size_in_bytes":7553},{"relative_path":"references/webhooks.md","size_in_bytes":3434}],"name":"openlayer","skill_md_contents":"---\nname: openlayer\ndescription: >-\n  Use when integrating, instrumenting, evaluating, or testing an app with Openlayer — the AI\n  evaluation and observability platform. Covers BOTH modes: (1) MONITORING/live — add tracing to\n  LLM or agent code with the openlayer Python or TypeScript SDK (@trace, trace_openai /\n  trace_anthropic / trace_litellm / trace_bedrock and the LangChain callback), publish inference\n  rows, enrich traces with user/session context, metadata, and guardrails; and (2) DEVELOPMENT/offline\n  — author openlayer.json + tests.json, push commits with the Openlayer CLI or MCP, gate CI/CD, and\n  run the fix-loop on failing rows. Also covers creating tests (the prebuilt catalog, thresholds),\n  inference pipelines, and programmatic data access. Trigger when the user mentions Openlayer,\n  OPENLAYER_API_KEY, OPENLAYER_INFERENCE_PIPELINE_ID, openlayer.json, tests.json, an inference pipeline,\n  \"openlayer push\", openlayer-mcp, trace_openai / trace_anthropic, Openlayer guardrails, or LLM evals.\nlicense: Apache-2.0\nallowed-tools:\n  - WebFetch(domain:docs.openlayer.com)\n  - WebFetch(domain:openlayer.com)\n  - Bash(openlayer whoami*)\n  - Bash(openlayer projects*)\n  - Bash(openlayer inspect*)\n  - Bash(openlayer validate*)\n  - Bash(curl *docs.openlayer.com/*)\n---\n\n# Openlayer\n\nThis skill helps you integrate apps with Openlayer correctly and fast — instrumenting code for live\nmonitoring, setting up offline evals, creating tests and guardrails, gating CI/CD, and accessing data.\n\n## Core Principles\n\nFollow these for ALL Openlayer work:\n\n1. **Docs-first.** NEVER implement from memory — the SDKs, CLI, MCP, and test catalog change often.\n   Resolve the current API from the docs before writing code. See `references/docs-access.md`.\n2. **Use the latest versions.** Install/upgrade `openlayer` (PyPI) / `openlayer` (npm) to latest; don't\n   downgrade to match an old snippet. See `references/sdk-upgrade.md`.\n3. **Identify the mode FIRST** — this is the primary routing decision:\n   - **Monitoring (live):** the user is shipping real traffic and wants traces/observability and\n     continuous tests on production data → `references/monitoring-instrumentation.md`.\n   - **Development (offline):** the user wants to evaluate a model/app version against datasets before\n     shipping, with results gating a commit/PR → `references/development-setup.md`.\n   - Unsure? Ask one question: \"Do you want to evaluate live production traffic, or test a version\n     offline before shipping?\" Many teams eventually use both.\n4. **Minimal footprint.** Wrap existing clients and decorate existing functions — don't rewrite app logic.\n5. **Never hardcode keys.** Use env vars: `OPENLAYER_API_KEY`, `OPENLAYER_BASE_URL` (self-hosted/local\n   only — the SDKs need it to **include** `/v1`, unlike the CLI profile URL; see `references/cli.md`),\n   `OPENLAYER_INFERENCE_PIPELINE_ID`, `OPENLAYER_DISABLE_PUBLISH`. Don't ask the user to paste\n   keys into chat — have them set the env var or a `.env`. Keys: Workspace settings → API keys.\n6. **Debugging is not rebuilding.** When something already set up isn't working, diagnose before\n   editing — most failures are an unset env var, a base URL, or an app that never loaded its env file,\n   not wrong instrumentation. Start from `references/troubleshooting.md`.\n\n## Data access (the \"API\" plane)\n\nWhen you need to read or modify Openlayer data programmatically, use this tiered fallback. Details and\nthe SDK resource map are in `references/data-access.md`.\n\n1. **SDK (default).** Use the typed client: `Openlayer(api_key=...)` (Python) or `new Openlayer({ apiKey })` (TS).\n2. **Raw REST** via the OpenAPI as a last resort.\n\n(If the Openlayer MCP server happens to be connected, prefer its tools — covered in a separate skill.)\n\n> The Openlayer **CLI is NOT the data-access tool** — it is the development/push workflow tool\n> (`openlayer push`, `validate`, `export`, …). See `references/cli.md`.\n\n## Documentation access\n\nOpenlayer docs at `docs.openlayer.com` are agent-friendly:\n\n- **Index:** fetch `https://docs.openlayer.com/llms.txt` — every page listed as `[title](url.md): description`.\n- **Read a page as markdown:** append `.md` to any docs URL (e.g. `https://docs.openlayer.com/monitoring/instrument.md`).\n- **Search:** the `search_openlayer_docs` MCP tool, and a hosted docs MCP at `docs.openlayer.com/mcp`.\n\nPreference order: search (when topic is fuzzy) → `llms.txt` lookup → fetch the specific `.md`. See\n`references/docs-access.md`.\n\n## Use case specific references\n\n| If the user wants to…                                   | Read                                      |\n| ------------------------------------------------------- | ----------------------------------------- |\n| **Debug a setup that isn't working** (no traces, push/413, auth, base URL) | `references/troubleshooting.md` |\n| Add live tracing / observability to code               | `references/monitoring-instrumentation.md` |\n| Monitor or evaluate a traditional / tabular ML model (scikit-learn, XGBoost, regression/classification) | `references/traditional-ml.md` |\n| Set up offline evals (`openlayer.json` / `tests.json`)  | `references/development-setup.md`          |\n| Create or choose tests / thresholds                     | `references/tests.md`                      |\n| Add runtime guardrails (block/redact PII, injection)    | `references/guardrails.md`                 |\n| Author a custom metric                                  | `references/custom-metrics.md`             |\n| Set up compliance frameworks / governance               | `references/governance.md`                 |\n| Route LLM calls through the Openlayer Gateway           | `references/gateway.md`                    |\n| Subscribe to platform events via webhooks               | `references/webhooks.md`                   |\n| Publish or stream inference rows directly               | `references/data-streaming.md`             |\n| Gate CI/CD on eval results                              | `references/ci-cd.md`                      |\n| Query data programmatically                             | `references/data-access.md`                |\n| Install/use the Openlayer CLI                           | `references/cli.md`                        |\n| Find or fetch docs                                      | `references/docs-access.md`                |\n| Install or upgrade the SDK                              | `references/sdk-upgrade.md`                |\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}