Update to DataRobot Skills

{
  "description": "Tools and guidance for model explainability, prediction explanations, feature impact analysis, SHAP values, SHAP distributions, anomaly assessment, and model diagnostics. Use when analyzing model explanations, feature impact, SHAP values, SHAP distributions, anomaly assessment, or diagnosing model behavior.\n",
  "included_files": [
    {
      "relative_path": "references/shap_api_reference.md",
      "size_in_bytes": 7558
    },
    {
      "relative_path": "references/xemp_pe_reference.md",
      "size_in_bytes": 5408
    },
    {
      "relative_path": "scripts/compute_shap_matrix.py",
      "size_in_bytes": 3336
    }
  ],
  "name": "datarobot-model-explainability",
  "skill_md_contents": "---\nname: datarobot-model-explainability\ndescription: >\n  Tools and guidance for model explainability, prediction explanations, feature impact analysis,\n  SHAP values, SHAP distributions, anomaly assessment, and model diagnostics. Use when analyzing\n  model explanations, feature impact, SHAP values, SHAP distributions, anomaly assessment, or\n  diagnosing model behavior.\n---\n\n# DataRobot Model Explainability Skill\n\nThis skill covers SHAP insights, XEMP prediction explanations, anomaly explanations, and model diagnostics.\n\n> **SDK version**: Use `datarobot>=3.6.0` for the full API set in this skill (`ShapDistributions`\n> was added in 3.6; `ShapMatrix`, `ShapImpact`, and `ShapPreview` are available in\n> `datarobot>=3.4.0`). Use `from datarobot.insights import ShapMatrix, ...` with\n> `entity_id=model_id` — not legacy `datarobot.models.ShapMatrix` (`project_id` / `dataset_id`).\n> `ShapMatrix`, `ShapImpact`, `ShapPreview`, and `ShapDistributions` are the canonical SHAP API.\n> The older `dr.PredictionExplanations` (XEMP-based) remains available but is the secondary path.\n\n---\n\n## Quick Start\n\n| Goal | API to use | Prerequisites |\n|------|-----------|---------------|\n| SHAP values for all features, all rows | `ShapMatrix.create(entity_id=model_id)` | None - universal SHAP |\n| Per-row top-feature explanations | `ShapPreview.create(entity_id=model_id)` | None |\n| Aggregated feature importance via SHAP | `ShapImpact.create(entity_id=model_id)` | None |\n| SHAP value distributions across features | `ShapDistributions.create(entity_id=model_id)` | None |\n| SHAP for a filtered segment | `dr.DataSlice.create(...)` + `ShapMatrix.create(..., data_slice_id=...)` | Data slice definition |\n| XEMP-based prediction explanations | `dr.PredictionExplanations.create(...)` | Feature Impact; PE initialization; dataset uploaded |\n| Anomaly explanations (time series) | `AnomalyAssessmentRecord.compute(project_id, model_id, ...)` | Anomaly model |\n| ROC / lift / confusion (insights) | `RocCurve.create(...)` / `LiftChart.create(...)` / `ConfusionMatrix.create(...)` | Validation data |\n| ROC / lift / confusion (Model helpers) | `model.get_roc_curve()` / `model.get_lift_chart()` / `model.get_confusion_chart()` | Validation data |\n\n**Universal SHAP is the preferred path** - no dataset pre-upload or Feature Impact step required.\n\n## When to use this skill\n\nUse this skill when you need to explain leaderboard model behavior, compute SHAP insights, use\nXEMP prediction explanations, analyze anomaly explanations, or retrieve model diagnostics.\n\n## Key capabilities\n\n### 1. SHAP insights\n\n- Compute `ShapMatrix`, `ShapPreview`, `ShapImpact`, and `ShapDistributions`\n- Filter insights with `dr.DataSlice`\n\n### 2. XEMP and anomaly explanations\n\n- Use XEMP `dr.PredictionExplanations` when specifically required\n- Retrieve time series anomaly assessment records and explanations\n\n### 3. Diagnostics\n\n- Retrieve ROC, lift, and confusion insights\n- Use Model helpers for ROC, lift, confusion, and feature effects\n\n## Setup\n\n```python\nimport datarobot as dr\nfrom datarobot.insights import ShapMatrix, ShapImpact, ShapPreview, ShapDistributions\n\ndr.Client()\n```\n\n---\n\n## Core API: `datarobot.insights`\n\n```python\nimport pandas as pd\nfrom datarobot.insights import ShapMatrix, ShapImpact, ShapPreview, ShapDistributions\n\nmodel_id = \"YOUR_MODEL_ID\"\n\nmatrix = ShapMatrix.create(entity_id=model_id)\ndf = pd.DataFrame(matrix.matrix, columns=matrix.columns)\n\nimpact = ShapImpact.create(entity_id=model_id)\npreview = ShapPreview.create(entity_id=model_id)\ndistributions = ShapDistributions.create(entity_id=model_id)\n```\n\nUse `ShapMatrix` for full row-by-feature SHAP values, `ShapPreview` for compact top-driver rows,\n`ShapImpact` for aggregated SHAP importance, and `ShapDistributions` for per-feature SHAP\ndistributions. Use `source=\"externalTestSet\"` plus `external_dataset_id` for external datasets.\nSee `references/shap_api_reference.md` for parameters, exports, and limitations.\n\n---\n\n## Secondary path: XEMP Prediction Explanations\n\nUse `dr.PredictionExplanations` when XEMP explanations are specifically required (e.g., certain\nregulatory contexts, or when SHAP is unavailable for the model type).\n\n**Prerequisites** (all required before calling `.create()`):\n1. Feature Impact must be computed: `model.request_feature_impact()` and wait\n2. Prediction explanations initialized: `dr.PredictionExplanationsInitialization.create(...)`\n3. Scoring dataset uploaded to the AI Catalog\n\n```python\nimport datarobot as dr\n\nmodel = dr.Model.get(project=project_id, model_id=model_id)\nmodel.request_feature_impact().wait_for_completion()\ndr.PredictionExplanationsInitialization.create(project_id=project_id, model_id=model_id)\n\ndataset = dr.Dataset.upload(\"./data/scoring_data.csv\")\npe_job = dr.PredictionExplanations.create(\n    project_id=project_id,\n    model_id=model_id,\n    dataset_id=dataset.id,\n    max_explanations=5,  # top N features per row, up to 50\n    threshold_high=0.5,  # only explain rows with prediction >= threshold\n    threshold_low=0.1,  # only explain rows with prediction <= threshold\n)\n\npe_obj = pe_job.get_result_when_complete()\n```\n\nUse `pe_obj.get_rows()`, `pe_obj.get_all_as_dataframe()`, or `pe_obj.download_to_csv(...)` to\nretrieve results. For parameters, multiclass modes, and exposure-adjusted predictions, see\n`references/xemp_pe_reference.md`.\n\n## Data slices for filtered insights\n\nUse `dr.DataSlice` when the user asks to explain model behavior for a segment, such as a\nregion, product line, target class, or high-risk cohort. Pass the resulting `data_slice_id` into\nthe `datarobot.insights` SHAP APIs.\n\n```python\nimport datarobot as dr\nfrom datarobot.insights import ShapMatrix\n\ndata_slice = dr.DataSlice.create(\n    name=\"high_income_customers\",\n    filters=[{\"operand\": \"income\", \"operator\": \">\", \"values\": 100000}],\n    project=project_id,\n)\n\nshap_matrix = ShapMatrix.create(\n    entity_id=model_id,\n    source=\"validation\",\n    data_slice_id=data_slice.id,\n)\n```\n\n---\n\n## Anomaly assessment (time series models)\n\nFor time series anomaly detection models, use `AnomalyAssessmentRecord`.\n\n```python\nfrom datarobot.models.anomaly_assessment import AnomalyAssessmentRecord\n\nrecord = AnomalyAssessmentRecord.compute(\n    project_id=project_id,\n    model_id=model_id,\n    backtest=0,  # backtest index (int) or \"holdout\"\n    source=\"validation\",  # \"training\" or \"validation\" only\n    series_id=None,  # required for multiseries projects\n)\n\nrecords = AnomalyAssessmentRecord.list(project_id=project_id, model_id=model_id)\nlatest = record.get_latest_explanations()\n\nregions = record.get_predictions_preview().find_anomalous_regions()\nexplanations = record.get_explanations_data_in_regions(regions=regions)\n\nranged = record.get_explanations(\n    start_date=\"2024-01-01T00:00:00.000000Z\",\n    end_date=\"2024-06-01T00:00:00.000000Z\",\n)\n```\n\n---\n\n## Model diagnostics\n\nUse the same `entity_id=model_id` pattern as SHAP insights. `FeatureEffects` / partial dependence\nis still retrieved through Model helpers (not in `datarobot.insights`).\n\n### Insights diagnostics (preferred — matches SHAP API)\n\n```python\nfrom datarobot.insights import RocCurve, LiftChart, ConfusionMatrix\n\nroc = RocCurve.create(entity_id=model_id)\nlift = LiftChart.create(entity_id=model_id)\nconfusion = ConfusionMatrix.create(entity_id=model_id)\n```\n\n### Model helpers (alternative)\n\n```python\nmodel = dr.Model.get(project=project_id, model_id=model_id)\n\nroc = model.get_roc_curve(source=\"validation\")\nlift = model.get_lift_chart(source=\"validation\")\nconfusion = model.get_confusion_chart(source=\"validation\")\n\n# Feature Impact (non-SHAP) and Feature Effects (partial dependence for top features)\nfi = model.get_feature_impact()\nfeature_effects = model.get_feature_effect(source=\"validation\")\n```\n\n---\n\n## Interpreting SHAP values\n\n- **Positive value**: feature pushes prediction higher than baseline\n- **Negative value**: feature pushes prediction lower than baseline\n- **Magnitude**: size of influence; larger absolute value = stronger effect\n- **Sum**: all SHAP values for a row sum to `prediction - base_value` in the link-function space\n- **`base_value`**: the model's mean prediction (the \"no information\" baseline)\n\nExample: if `base_value = 0.35` and a row's prediction is `0.72`, the row's SHAP values sum to\n`0.37` when `link_function = \"identity\"`. A feature with SHAP `+0.20` contributed 20 units in\nthat same link-function space above baseline.\n\nWhen `link_function = \"logit\"`, SHAP values are in log-odds space. Add feature contributions to\n`base_value` in log-odds space, then use inverse-logit (`scipy.special.expit`) on the resulting\ntotal to convert it to a probability. Do not apply `expit` to individual SHAP values as if they\nwere probability deltas.\n\n---\n\n## Decision guide\n\n```\nTask: explain predictions\n    |\n    - Need all features + all rows?     -> ShapMatrix.create(entity_id=model_id)\n    - Need top-N features per row?      -> ShapPreview.create(entity_id=model_id)\n    - Need aggregated importance?       -> ShapImpact.compute(entity_id=model_id)\n    - Need feature SHAP distributions?  -> ShapDistributions.create(entity_id=model_id)\n    - Need a segment/cohort only?       -> dr.DataSlice + data_slice_id\n    - XEMP required (regulatory/type)?  -> dr.PredictionExplanations.create(...)\n    - Time series / anomaly model?      -> AnomalyAssessmentRecord.compute(project_id, model_id, ...)\n```\n\n---\n\n## Common errors\n\n| Error | Cause | Fix |\n|-------|-------|-----|\n| `SHAP not available for this model` | Unsupported model type, or anomaly-detection model with >1000 features | Check model support; use XEMP PE if SHAP is unavailable |\n| `Feature Impact not computed` | PredictionExplanations prerequisite missing | Run `model.request_feature_impact()` and wait |\n| Missing `PredictionExplanationsInitialization` | PE not initialized | Call `PredictionExplanationsInitialization.create()` |\n| `source='holdout'` fails | Holdout not unlocked | Unlock holdout in project settings first |\n| Empty `previews` | No rows in partition | Check partition contains data |\n\n---\n\n## Reference files\n\n- `references/shap_api_reference.md` - full parameter signatures for ShapMatrix, ShapImpact, ShapPreview, ShapDistributions\n- `references/xemp_pe_reference.md` - PredictionExplanations and PredictionExplanationsInitialization parameter reference\n- `scripts/compute_shap_matrix.py` - compute and export ShapMatrix to CSV or DataFrame\n\n## Resources\n\n- [datarobot.insights API reference](https://datarobot-public-api-client.readthedocs-hosted.com/en/latest-release/insights.html)\n- [Prediction Explanations user guide](https://datarobot-public-api-client.readthedocs-hosted.com/en/latest-release/reference/modeling/insights/prediction_explanations.html)\n"
}