Update to RAG & GenAI Copilot

{
  "description": "Design traces, logging, metrics, dashboards, and SLO monitoring for production retrieval and grounded-answer systems.",
  "included_files": [],
  "name": "observe-rag",
  "skill_md_contents": "---\nname: observe-rag\ndescription: Design traces, logging, metrics, dashboards, and SLO monitoring for production retrieval and grounded-answer systems.\n---\n\n\n# Observe RAG behavior\n**Trigger:** Missing traces, difficult debugging, monitoring, SLOs, operational dashboards or incident detection.\n\n1. Define unit of observation: query/request ID, user/tenant reference (minimized), corpus/index version, retrieval configuration, model/config version.\n2. Instrument ingestion lag, parser failures, index counts, ACL-filtered retrieval, candidate IDs/scores, reranking, packed source IDs, answer/citations and errors.\n3. Record stage latency p50/p95/p99 as justified, token usage, retries, timeouts, cost, answerable/abstained counts and safe quality samples.\n4. Design privacy: redact prompts, secrets and content by default; choose retention and access policy; sampling must not expose private source text.\n5. Create a minimal request-to-evidence trace and a dashboard with leading indicators and actionable alerts. Separate operational SLOs from offline quality evals.\n6. Output instrumentation map, sample trace schema, dashboard/alert plan and evidence needed to validate it.\n\nRead `../../references/observability-slo.md` and `../../templates/trace-record.json`.\n"
}