Update to Tuist

{
  "description": "Analyzes Xcode selective testing effectiveness for a test run, showing which test targets were skipped or ran, and diagnosing regressions in test selection. Can compare two test runs to identify what changed.",
  "included_files": [],
  "name": "analyze-selective-testing",
  "skill_md_contents": "---\nname: analyze-selective-testing\ndescription: Analyzes Xcode selective testing effectiveness for a test run, showing which test targets were skipped or ran, and diagnosing regressions in test selection. Can compare two test runs to identify what changed.\n---\n\n# Analyze Selective Testing\n\n## Quick Start\n\n1. Run `tuist test list --json` to find test runs.\n2. Run `tuist test xcode target list <test-run-id> --json` to see per-target selective testing status.\n3. If effectiveness dropped, compare targets between a known-good run and the regressed run.\n\n## Step 1: Resolve Test Runs\n\n### Find the test run to analyze\n\nList recent test runs, optionally filtering by branch:\n\n```bash\ntuist test list --json --page-size 10\ntuist test list --git-branch feature-x --json --page-size 5\n```\n\nGet basic info for a specific test run:\n\n```bash\ntuist test show <test-run-id> --json\n```\n\nUse this to confirm the run's branch, status, scheme, and environment (`xcode_version`, `macos_version`).\n\n### Defaults\n\n- If no test run is specified, use the most recent CI test run on the current branch.\n- For comparison, use the most recent CI test run on the project's default branch.\n\n## Step 2: Drill Into Selective Testing Targets\n\nList per-target selective testing data for a test run:\n\n```bash\ntuist test xcode target list <test-run-id> --json\n```\n\nEach target includes:\n- **name**: The test target name\n- **hit_status**: `miss` (ran), `local` (skipped via local cache), or `remote` (skipped via remote cache)\n- **hash**: The selective testing hash for this target\n\nFilter by status to focus your analysis:\n\n```bash\ntuist test xcode target list <test-run-id> --hit-status miss --json\ntuist test xcode target list <test-run-id> --hit-status local --json\ntuist test xcode target list <test-run-id> --hit-status remote --json\n```\n\n### Assess effectiveness\n\nCount the targets by status:\n```\neffectiveness = (local_hits + remote_hits) / total_targets * 100\n```\n\n| Effectiveness | Verdict |\n|---|---|\n| 60-100% | Healthy — most unchanged tests are being skipped |\n| 30-60% | Moderate — some cache invalidation occurring, worth investigating |\n| 0-30% | Low — likely a core dependency changed or the cache was invalidated |\n| 0% | Cold cache — first run, environment change, or full invalidation |\n\n## Step 3: Diagnose Regressions\n\nIf effectiveness dropped, investigate:\n\n### Check for global invalidation\n\nIf nearly all targets show as `miss`, the entire selective testing cache was invalidated. Common causes:\n\n| Cause | How to verify |\n|---|---|\n| Xcode version change | Compare `xcode_version` between good and bad runs via `tuist test show` |\n| CI environment change | Compare `macos_version`, `model_identifier` between runs |\n| Project graph or dependency change | Check git history for changes to project manifests, dependency versions, or target configurations |\n| Core dependency change | A widely-depended-on target changed, invalidating all its dependents |\n\nNote: Tuist CLI version upgrades rarely cause hash invalidation — the hash version is not updated on every release.\n\n### Check for cascade invalidation\n\nIf only some targets show as `miss`, a dependency change likely cascaded:\n\n1. Run `tuist test xcode target list <good-run-id> --json` and `tuist test xcode target list <bad-run-id> --json`.\n2. Match targets by name — identify those that changed from `local`/`remote` to `miss`.\n3. Compare hashes for changed targets — if a target's hash differs, its sources or dependencies changed.\n4. Look for a common dependency among the invalidated targets.\n\n### Compare two test runs\n\nFor each target:\n- Match by `name`\n- Compare `hit_status` (hit -> miss = invalidated, miss -> hit = improved)\n- Compare `hash` (different hash = target or its dependencies changed)\n\n## Step 4: Recommendations\n\nBased on the diagnosis:\n\n- **Environment change**: Ensure CI uses consistent Xcode/macOS versions. This is the most common cause of full cache invalidation.\n- **Cascade from dependency change**: Consider if the change to the root target was necessary. If the root target is a widely-shared module, consider splitting it.\n- **Cold cache on new branch**: Run tests once to warm the cache. Subsequent runs will benefit from selective testing.\n\n## Summary Format\n\nProduce a summary with:\n\n1. **Overall**: effectiveness percentage, verdict (healthy/moderate/low/cold)\n2. **Target breakdown**: targets grouped by hit status\n3. **Comparison** (if applicable): targets that changed status between runs, with hash diffs\n4. **Root cause**: what caused the regression\n5. **Recommendations**: actionable next steps\n\nExample:\n\n```\nSelective Testing Analysis: test run abc123 on feature-x\n\nEffectiveness: 15% (3/20 targets skipped) -- LOW\n  Local hits:  2 targets (CoreTests, UtilTests)\n  Remote hits: 1 target (NetworkTests)\n  Misses:      17 targets\n\nComparison with baseline (run def456, 75% effectiveness):\n  14 targets changed from hit to miss\n  All changed targets have different hashes\n\nRoot cause: Xcode version changed from 15.2 to 16.0 between runs.\nThis changed all target hashes, invalidating the entire cache.\n\nRecommendations:\n- Align CI Xcode version with the version used in the baseline run.\n- If the upgrade is intentional, run tests once to re-warm the cache.\n```\n\n## Done Checklist\n\n- Identified the test run(s) to analyze\n- Drilled into per-target selective testing status via `tuist test xcode target list`\n- Diagnosed root cause if effectiveness is low\n- Compared targets with baseline if regression detected\n- Provided actionable recommendations\n"
}