Update to RAG & GenAI Copilot

{
  "description": "Build measurable RAG evaluations for retrieval relevance, grounding, citations, abstention, security, latency, and cost.",
  "included_files": [],
  "name": "evaluate-rag",
  "skill_md_contents": "---\nname: evaluate-rag\ndescription: Build measurable RAG evaluations for retrieval relevance, grounding, citations, abstention, security, latency, and cost.\n---\n\n\n# Design and run RAG evaluations\n**Trigger:** Evaluation design, regression gates, quality scoring, testing retrieval settings or checking an architecture change.\n\n1. Name the product decision: compare retrievers, catch regressions, choose a model, validate authorization or accept a launch.\n2. Build a versioned dataset with query, expected evidence IDs, answerable/no-answer label, tenant/scopes, source version, language and category; avoid synthetic-only data.\n3. Separate retrieval (recall@k, MRR, nDCG), context (evidence present after packing), generation (groundedness, relevance, completeness), citation claim-support, abstention, access/security and cost/latency.\n4. Define metric denominators, missing-evidence handling and per-slice thresholds. Pair automatic judges with targeted human review; validate judge reliability.\n5. Establish baseline/holdout, paired comparisons, uncertainty and regression gates. Do not cherry-pick cases or claim live run without artifacts.\n6. Execute only with actual tools/data. For deterministic, provided JSON fixtures, `scripts/score_retrieval.py` can compute basic metrics; it cannot measure groundedness or security.\n7. Deliver evaluation spec, data manifest, result table if run, failure slices, deployment gate and follow-up experiments.\n\nRead `../../references/evaluation-metrics.md`, `../../references/evidence-contract.md` and `../../templates/eval-plan.md`.\n"
}