{"id":77100,"plugin_id":"plugins_6ab42163e9048191ab6e8c14258a48c3","kind":"skill","collection_source":"plugin_package","comparison_source":null,"observed_at":"2026-11-03T00:12:26.431Z","digest":"d8ce309e1d50fec0c538024b2682373db7aed5dd44f31f570569c7c7d6b4e676","against":null,"payload":{"description":"Design traces, logging, metrics, dashboards, and SLO monitoring for production retrieval and grounded-answer systems.","included_files":[],"name":"observe-rag","skill_md_contents":"---\nname: observe-rag\ndescription: Design traces, logging, metrics, dashboards, and SLO monitoring for production retrieval and grounded-answer systems.\n---\n\n\n# Observe RAG behavior\n**Trigger:** Missing traces, difficult debugging, monitoring, SLOs, operational dashboards or incident detection.\n\n1. Define unit of observation: query/request ID, user/tenant reference (minimized), corpus/index version, retrieval configuration, model/config version.\n2. Instrument ingestion lag, parser failures, index counts, ACL-filtered retrieval, candidate IDs/scores, reranking, packed source IDs, answer/citations and errors.\n3. Record stage latency p50/p95/p99 as justified, token usage, retries, timeouts, cost, answerable/abstained counts and safe quality samples.\n4. Design privacy: redact prompts, secrets and content by default; choose retention and access policy; sampling must not expose private source text.\n5. Create a minimal request-to-evidence trace and a dashboard with leading indicators and actionable alerts. Separate operational SLOs from offline quality evals.\n6. Output instrumentation map, sample trace schema, dashboard/alert plan and evidence needed to validate it.\n\nRead `../../references/observability-slo.md` and `../../templates/trace-record.json`.\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}