Update to BugBug

{
  "description": "Validates and debugs BugBug test runs. Use to inspect an existing or latest test run, classify failures, inspect logs, DOM snapshots, screenshots, focus on a failed step run, or find a supported fix.",
  "included_files": [
    {
      "relative_path": "agents/openai.yaml",
      "size_in_bytes": 380
    },
    {
      "relative_path": "assets/bugbug-icon.svg",
      "size_in_bytes": 376
    },
    {
      "relative_path": "references/assertion-troubleshooting.md",
      "size_in_bytes": 1595
    },
    {
      "relative_path": "references/selectors-troubleshooting.md",
      "size_in_bytes": 4559
    },
    {
      "relative_path": "references/variables-troubleshooting.md",
      "size_in_bytes": 8033
    },
    {
      "relative_path": "references/visual-regression-review.md",
      "size_in_bytes": 6001
    },
    {
      "relative_path": "references/waiting-conditions-results-check.md",
      "size_in_bytes": 6516
    }
  ],
  "name": "bugbug-test-run-debugging",
  "skill_md_contents": "---\nname: bugbug-test-run-debugging\ndescription: Validates and debugs BugBug test runs. Use to inspect an existing or latest test run, classify failures, inspect logs, DOM snapshots, screenshots, focus on a failed step run, or find a supported fix.\n---\n\n# BugBug Test Run Debugging\n\n## Preflight\n\n1. Run `bugbug-context-discovering` before inspecting or repairing test runs.\n2. If a repair may change DOM-targeted steps, load\n   `bugbug-selectors-authoring`.\n3. Confirm the project, test run, test, or step run context before diagnosis.\n\n## Inputs\n\nResolve at least one of:\n\n- `testRunId` (preferred) to inspect a specific run.\n- `testId` to inspect the latest run for that test.\n- `stepRunId` to focus on a specific step run within the selected run.\n\n## Safety\n\n**The iron law: no repair without run evidence first.** A failed step tells you\n*where* the run stopped, not *why*. Until Phases 1 and 2 of the `Workflow` are\ncomplete, do not propose or apply a repair. Rewriting a selector because a step\nfailed on a selector is a symptom fix.\n\n- Do not debug suite runs with this skill. Use test-level runs only.\n- Do not start a new run just because a `testId` was provided. Inspect the latest\n  run first.\n- Do not mutate a test unless the user explicitly requests the specific repair.\n- Treat `stepRunId` as an optional focus, not a required failure identifier.\n- Stop after two failed repairs on the same step. Question the step instead.\n- Stop when no project, test, run, or step evidence can be confirmed. Ask for the\n  missing identifier or selection instead of guessing.\n\n## Workflow\n\n```mermaid\nflowchart TD\n    A[\"Phase 1: Gather evidence<br>(run payload, failed step, artifacts)\"] --> B{\"Reproducible?\"}\n    B -->|\"passes on re-run\"| C([\"Classify as flaky<br>report timing/state cause\"])\n    B -->|\"fails consistently\"| D[\"Phase 2: Compare<br>vs. last passing run\"]\n    B -->|\"cannot re-run\"| D\n    D --> E[\"Classify failure<br>(see Failure routing)\"]\n    E --> F[\"Phase 3: State ONE hypothesis<br>'X is root cause because Y'\"]\n    F --> G{\"Evidence confirms it?\"}\n    G -->|no| H[\"Discard, form new hypothesis\"]\n    H --> F\n    G -->|yes| I{\"Repair authorized<br>by Safety?\"}\n    I -->|no| J([\"Stop<br>recommend smallest repair\"])\n    I -->|yes| K[\"Phase 4: Apply ONE<br>smallest change\"]\n    K --> L[\"Re-run and verify\"]\n    L --> M{\"Step passes?\"}\n    M -->|yes| N([\"Report fix & evidence\"])\n    M -->|\"no, attempts < 2\"| F\n    M -->|\"no, attempts >= 2\"| O([\"Stop<br>question the step, not the selector\"])\n```\n\nComplete each phase before starting the next.\n\n### Phase 1: Gather evidence\n\n1. Inspect the provided run with `bugbug_get_test_run`. If only `testId` is\n   available, find its latest run with `bugbug_list_test_runs` first.\n2. Treat the returned run payload as the default evidence source. Follow its\n   artifact and screenshot links only when more evidence is needed. Never\n   construct run resource URIs manually.\n3. Read the error code and message on the failed step before forming any opinion.\n   The code (`ELEMENT_DOES_NOT_EXIST`, `FAILED_WAITING_CONDITIONS`, and similar)\n   already narrows the routing.\n4. Focus on the failed step with `bugbug_get_failed_step_run_details`.\n5. Establish reproducibility. Check earlier runs of the same test: does this step\n   fail every time, or intermittently? An intermittently failing step is a timing\n   or state problem, and repairing its selector will not fix it.\n\n### Phase 2: Compare against a passing run\n\n1. Find the most recent passing run of the same test with\n   `bugbug_list_test_runs` (filter by `status=passed`).\n2. Compare the failed step against its last passing execution: selector used,\n   waiting conditions, resolved variables, and page state.\n3. List every difference before deciding which one matters. Do not assume a\n   difference is irrelevant because it looks cosmetic.\n4. Check what changed around the run: app deployment, test edits, profile or\n   variable changes, environment.\n5. Classify the failure using **Failure routing** below.\n\n### Phase 3: Form one hypothesis\n\n1. State it explicitly: \"I think X is the root cause because Y.\" Name the\n   evidence Y refers to.\n2. Verify the hypothesis against the DOM snapshot, screenshots, or run logs\n   before touching the test. `bugbug_query_dom_snapshot_element` confirms whether\n   a selector resolves in the captured DOM.\n3. If the evidence contradicts the hypothesis, discard it and form a new one. Do\n   not stack a second theory on top of the first.\n4. If you cannot form an evidence-backed hypothesis, say so and report what\n   additional evidence is needed. Do not guess.\n\n### Phase 4: Repair and verify\n\nOnly enter this phase when the user has explicitly authorized the specific\nrepair. Otherwise stop and recommend it.\n\n1. Apply the single smallest change that addresses the confirmed root cause. No\n   bundled cleanups, no \"while I'm here\" edits to neighbouring steps.\n2. Validate with a new run and confirm the previously failing step now passes.\n3. Confirm no later step broke as a result.\n4. If the step still fails, count your attempts:\n   - **Fewer than 2 repairs tried**: return to Phase 3 with the new run's\n     evidence and form a different hypothesis.\n   - **2 or more repairs tried**: stop. Do not attempt another repair.\n\n### After 2 failed repairs: question the step\n\nRepeated failed repairs on the same step usually mean the step itself is wrong,\nnot its selector. Stop and ask:\n\n- Is this the right step type for the interaction?\n- Is a `switchContext` step missing for an iframe, or shadow-host selectors\n  missing from the preset?\n- Is a required waiting condition missing, or is an active one meaningless for\n  this UI state?\n- Did an earlier step fail to reach the expected application state, making this\n  step unreachable as written?\n- Is the test design itself the problem — missing setup, wrong profile, or a flow\n  that no longer exists in the product?\n\nReport this to the user as a design question. Do not attempt repair #3.\n\n### Failure routing\n\n- **Selector problem**: Confirm the active selector is broken, compare it with\n  the last passing run, and use screenshot plus DOM evidence for replacements.\n- **Visual regression**: Compare expected, observed, and diff screenshots; decide\n  whether to report a regression or update the baseline.\n- **Assertion problem**: Explain expected versus observed behavior and where the\n  fix belongs.\n- **Waiting conditions problem**: Inspect failed and skipped waiting-condition\n  results, then locate the fix in test state, app state, data, environment, or\n  waiting-condition configuration.\n- **Test data or variables problem**: Identify missing or invalid variables,\n  credentials, tenant data, or preconditions. Check whether the variable was\n  unresolved, empty, or wrong before repairing the step that visibly failed. Do\n  not invent secrets.\n- **App or product problem**: Cite the run evidence and report the product\n  behavior that changed.\n- **Environment problem**: Cite network, console, timeout, region, profile, or\n  infrastructure clues.\n\n### Red flags\n\nIf you catch yourself thinking any of these, return to Phase 1.\n\n| Thought | Reality |\n|---------|---------|\n| \"The step failed on a selector, so rewrite the selector\" | The error code names the symptom. A resolving selector on a wrong page is still a failure. |\n| \"Just bump the timeout and re-run\" | Timeout increases are valid only when evidence shows the condition eventually resolves. |\n| \"Let me fix these three steps at once\" | You cannot tell which change worked, and you may mask a second cause. |\n| \"It failed once, that's enough to diagnose\" | Check reproducibility first. Intermittent failures are timing or state, not selectors. |\n| \"I'll re-run it and see what happens\" | Re-running without a hypothesis is guessing with extra steps. |\n| \"One more repair attempt\" (after 2) | 2+ failures means the step or test design is wrong. Question it instead. |\n| \"The user will want this other cleanup too\" | Repairs are authorized individually. Recommend, don't apply. |\n\n## Output\n\n- Selected test run and failed-step context.\n- Triage confidence\n- Summary\n- Evidence-backed failure classification.\n- The hypothesis you confirmed, and the evidence confirming it.\n- Affected files, if tested app codebase available\n- Smallest proposed solution or next diagnostic action.\n- Verification actions\n- Explicit blockers, uncertainty, and any repair requiring user confirmation.\n\n## Resources\n\nResolve bundled resource paths relative to this skill directory.\n\n- `assets/bugbug-icon.svg` is the OpenAI UI icon for this skill.\n- Load `bugbug-context-discovering` during Preflight.\n- Read `references/selectors-troubleshooting.md` for selector failure diagnosis\n  and repair.\n- Read `references/assertion-troubleshooting.md` for assertion failure diagnosis\n  and repair.\n- Read `references/visual-regression-review.md` for screenshot comparison and baseline decisions.\n- Read `references/waiting-conditions-results-check.md` for\n  `FAILED_WAITING_CONDITIONS` diagnosis and repair decisions.\n- Read `references/variables-troubleshooting.md` for unresolved, empty, or wrong\n  variable diagnosis, including profile scope and secret placeholders.\n- Load `bugbug-selectors-authoring` before changing DOM-targeted steps.\n"
}