Update to Tuist

{
  "name": "compare-test-case",
  "description": "Compares a single test case's behavior across two branches, analyzing pass/fail status, duration, flakiness, and failure details. Useful for investigating test regressions introduced by a feature branch.",
  "included_files": [],
  "skill_md_contents": "---\nname: compare-test-case\ndescription: Compares a single test case's behavior across two branches, analyzing pass/fail status, duration, flakiness, and failure details. Useful for investigating test regressions introduced by a feature branch.\n---\n\n# Compare Test Case\n\n## Quick Start\n\nYou'll typically receive a test case identifier and two branches. Follow these steps:\n\n1. Run `tuist test case show <id-or-identifier> --json` to get the test case metrics.\n2. Run `tuist test case run list <identifier> --json` to see runs across branches.\n3. Compare behavior between base and head branches.\n4. Inspect failures with `tuist test case run show <run-id> --json`.\n5. Summarize findings with root cause analysis.\n\n## Step 1: Resolve the Test Case\n\n### By ID or dashboard URL\n\n```bash\ntuist test case show <test-case-id> --json\n```\n\n### By identifier (Module/Suite/TestCase)\n\n```bash\ntuist test case show Module/Suite/TestCase --json\n```\n\n### If no test case is provided\n\nDiscover flaky or failing tests to investigate:\n\n```bash\ntuist test case list --flaky --json --page-size 10\n```\n\nKey fields from the response:\n- `id` -- unique identifier for subsequent commands\n- `name`, `module_name`, `suite_name` -- the test identity\n- `reliability_rate` -- percentage of successful runs\n- `flakiness_rate` -- percentage of flaky runs in the last 30 days\n- `total_runs` / `failed_runs` -- volume context\n- `is_flaky` / `is_quarantined` -- current flags\n\n## Step 2: Get Runs on Each Branch\n\nList test case runs filtered by the test case, and look at the `git_branch` field:\n\n```bash\ntuist test case run list <identifier> --json --page-size 20\n```\n\nSeparate runs by branch. For each branch, compute:\n- Pass rate: `passed_runs / total_runs * 100`\n- Average duration\n- Flaky run count\n- Most recent status\n\n### Defaults\n\n- If no base branch is provided, use the project's default branch (usually `main`).\n- If no head branch is provided, detect the current git branch.\n\n## Step 3: Compare Branch Behavior\n\n| Metric | Base branch | Head branch | Verdict |\n|---|---|---|---|\n| Pass rate | e.g. 100% | e.g. 60% | REGRESSION |\n| Avg duration | e.g. 0.5s | e.g. 2.1s | REGRESSION |\n| Flaky runs | 0 | 3 | NEW FLAKINESS |\n| Last status | success | failure | REGRESSION |\n\nClassify the change:\n- **Newly failing**: 100% pass rate on base, <100% on head\n- **Newly flaky**: No flaky runs on base, flaky runs on head\n- **Duration regression**: >50% increase in average duration\n- **Fixed**: Failing on base, passing on head\n- **Stable**: Same behavior on both branches\n\n## Step 4: Inspect Failures\n\nFor each failing run on the head branch:\n\n```bash\ntuist test case run show <test-case-run-id> --json\n```\n\nExamine:\n- `failures[].message` -- the assertion or error message\n- `failures[].path` -- source file path\n- `failures[].line_number` -- exact line of failure\n- `failures[].issue_type` -- type of issue\n- `repetitions` -- shows retry behavior (e.g., pass-fail-pass means flaky)\n- `crash_report` -- crash data if the test runner crashed\n\n## Step 5: Identify Root Cause\n\nBased on the comparison:\n\n### Newly failing\n- Check commits between base and head branches for changes to the test file or the code under test.\n- Look at the failure message for clues about what changed.\n\n### Newly flaky\n- Common patterns: timing/async issues, shared state, environment dependencies.\n- Check if `repetitions` show intermittent pass/fail patterns.\n- See the fix-flaky-tests skill for detailed flaky test analysis patterns.\n\n### Duration regression\n- Check if setup/teardown time increased.\n- Check if the test is doing more work (new assertions, larger data sets).\n- Check if a dependency became slower.\n\n## Summary Format\n\nProduce a summary with:\n\n1. **Test case info**: Name, module, suite, overall reliability.\n2. **Base branch behavior**: Pass rate, avg duration, flaky count.\n3. **Head branch behavior**: Pass rate, avg duration, flaky count.\n4. **Verdict**: What changed and classification.\n5. **Root cause**: Hypothesis based on failure analysis.\n6. **Recommendations**: Specific file paths, line numbers, and fix suggestions.\n\nExample:\n\n```\nTest Case Comparison: AuthModuleTests/LoginTests/test_login_with_expired_token\n\nOverall reliability: 85% (was 100% before head branch)\n\nBase (main):\n  Pass rate: 100% (15/15 runs)\n  Avg duration: 0.3s\n  Flaky: No\n\nHead (feature/auth-refactor):\n  Pass rate: 60% (3/5 runs)\n  Avg duration: 0.5s\n  Flaky: Yes (2 flaky runs)\n\nVerdict: NEWLY FLAKY -- test was stable on main but intermittently fails on feature branch\n\nRoot cause: The auth refactor introduced an async token refresh that races with the\ntest's synchronous assertion. Failures show \"Expected status 401, got nil\" at\nTests/AuthModuleTests/LoginTests.swift:42, suggesting the response arrives before\nthe token refresh completes.\n\nRecommendations:\n- Add an await/expectation before the assertion at LoginTests.swift:42\n- Consider mocking the token refresh to make the test deterministic\n```\n\n## Done Checklist\n\n- Resolved the test case identity\n- Gathered runs on both branches\n- Compared pass rates, durations, and flakiness\n- Inspected failure details for failing runs\n- Identified root cause with file paths and line numbers\n- Provided actionable fix recommendations\n"
}