{"id":20050,"plugin_id":"plugins_6aa24265df788191a25e7db0cdeca894","kind":"skill","collection_source":null,"comparison_source":null,"observed_at":"2026-09-30T23:15:59.907Z","digest":"7fbd110d0ea7e820b60f77af18a7464af1756778035fba8b1ebac3ff2e09f08c","against":null,"payload":{"name":"ansight-ui-testing","description":"Initialize, author, extract, debug, validate, run, or maintain source-controlled Ansight workspace automation. Use for tests, tasks, triggers, trends, sanitizers, schemas, or workspace registration; do not use for routine live interaction, session investigation, or readiness scoring unless needed to build or verify a workspace asset.","included_files":[{"relative_path":"agents/openai.yaml","size_in_bytes":307}],"skill_md_contents":"---\nname: ansight-ui-testing\ndescription: Initialize, author, extract, debug, validate, run, or maintain source-controlled Ansight workspace automation. Use for tests, tasks, triggers, trends, sanitizers, schemas, or workspace registration; do not use for routine live interaction, session investigation, or readiness scoring unless needed to build or verify a workspace asset.\n---\n\n## OpenAI plugin integration\n\nRun Ansight CLI commands using the execution tools in ChatGPT Work or Codex on the machine that owns the resident host. If this environment cannot execute commands or reach that host, explain the missing prerequisite and do not claim a live inspection succeeded. A remote workspace or cloud agent does not automatically have access to the developer’s local host. Resolve relative helper paths from this skill’s directory. When this workflow references another bundled skill, read its local SKILL.md completely before following it.\n\nUse these bundled files for the canonical skill URLs referenced below; keep public URLs when writing documentation for the user’s app:\n\n- https://www.ansight.ai/skills/ansight-cli-setup.md → [ansight-cli-setup](../ansight-cli-setup/SKILL.md)\n- https://www.ansight.ai/skills/ansight-install.md → [ansight-install](../ansight-install/SKILL.md)\n\n\n# Ansight Workspace Automation\n\nBuild and maintain the repository-owned `ansight/` playbook for one app. Prefer the Ansight CLI for workspace initialization, scaffolding, discovery, execution, and validation. Edit the generated source files when their behavior or policy must be implemented.\n\n## Understand The Workspace Components\n\n| Component | What it accomplishes | Why use it |\n| --- | --- | --- |\n| Test (`ansight/tests/**/*.json`) | Gives a bounded agent a natural-language user journey and observable final-state validation. | Use it when the path requires observation or decisions and success can be proved through Ansight evidence. |\n| Task (`ansight/tasks/**/*.ts`) | Encodes an explicitly invoked, deterministic sequence of known tool calls with named assertions. | Use it to make a stable app-specific workflow faster, cheaper, and repeatable without asking an agent to rediscover every step. |\n| Trigger (`ansight/triggers/**/*.ts`) | Reacts automatically to one matching app or session event and may request one bounded app action. | Use it to enrich future session evidence at a meaningful moment, such as capturing app-owned diagnostics after an error. Do not use it as a scheduler or general background worker. |\n| Trends (`ansight/trends/**/*.json`) | Defines a stable SDK-event observation span, calculates deterministic telemetry metrics, evaluates fixed budgets, and optionally compares each metric with comparable historical runs. | Use it for repeatable FPS, memory, duration, evidence, or other numeric trend rules and regression monitoring rather than subjective agent judgment. |\n| Sanitizer (`ansight/sanitizers/**/*.ts`) | Redacts, transforms, or removes sensitive content while Ansight creates a separate session archive. | Use it before exporting or sharing captured data; it does not modify the original session. |\n| Schemas and support files (`ansight/schema`, declarations, and `tsconfig.json`) | Provide versioned contracts, editor completion, and local or CI validation. | Use them to author against the installed host contract and catch invalid definitions before discovery or execution. They are not runtime workflows. |\n\nA test is agentic; a task is deterministic; a trigger is event-driven; Trends turns event instrumentation into an observation span, metric budgets, and optional regression monitoring; and a sanitizer protects an exported copy.\n\n## Prerequisites And Scope\n\nAuthoring and validating a workspace require the Ansight CLI, but they do not require a connected app. If the executable or local host is not ready, follow `https://www.ansight.ai/skills/ansight-cli-setup.md`.\n\nRunning a test or task, verifying a trigger against live events, or capturing new evidence requires a development or QA build with an initialized Ansight SDK integration and the matching App ID. If the integration is missing, follow `https://www.ansight.ai/skills/ansight-install.md`. Do not require the SDK merely to edit definitions or extract a draft from an existing recorded session.\n\nWorkspace authoring does not authorize execution. Run a test or task, connect a trigger, rebuild Trends history, export a sanitized archive, or synchronize cloud history only when the user requested that state-changing operation or verification.\n\nFor workspace automation requests, prefer existing SDK and host capabilities. Verify their execution, transfer, and persistence behavior before concluding that app changes are required. An unverified capability is an uncertainty, not evidence that a new tool is needed.\n\nPreserve the requested outcome: copying a file does not automatically require a transactional snapshot or custom artifact provider. If existing capabilities demonstrably cannot satisfy the request, explain the specific gap and propose app integration work separately, unless that work is already authorized.\n\n## Inspect And Initialize With The CLI\n\nResolve the repository root and exact App ID, then inspect the existing `ansight/` tree before adding anything:\n\n```sh\nansight app list --json\nansight app get --json\n```\n\nInitialize missing workspace directories and canonical support files with:\n\n```sh\nansight workspace init --app-id --json\n```\n\nRegistration links the App ID to the trusted codebase and enables automatic Trends evaluation when matching sessions finalize. Use `--no-register` only when the user wants scaffolding without that link. Initialization is idempotent and preserves customized files; do not use `--force` unless replacing support files is explicitly intended.\n\nWhen a workspace already exists but its app registration must be added or repaired, use:\n\n```sh\nansight app register --codebase --json\n```\n\n## Scaffold Before Editing\n\nUse the CLI generators when they exist so collision checks, canonical declarations, and current defaults come from the installed host:\n\n```sh\nansight workspace add test --app-id --json\nansight workspace add task --app-id --json\nansight workspace add trigger --app-id --json\nansight workspace add sanitizer --json\n```\n\nThen edit the generated definition. Existing files fail safely unless `--force` is supplied; do not replace one silently. There is no `workspace add trends` command, so create strict JSON files from the current [Trends contract](https://www.ansight.ai/docs/workspace/trends).\n\nDerive stable IDs from paths by removing the extension and replacing directory separators with dots. Preserve local naming conventions. Use contract version `1`, strict portable JSON, and the canonical declarations produced by the installed host. Do not invent runtime APIs or schema fields.\n\n## Tests: Agentic Journeys\n\nRead the current [Workspace Tests](https://www.ansight.ai/docs/workspace/tests) contract before authoring. Keep actions and journey context in `prompt`; keep observable success criteria in `validation`. Require the exact `appId`. Put only secret aliases in `requiredSecrets`, never secret values. Trends evaluation is session-driven and independent of workspace test outcomes.\n\nList and validate before execution:\n\n```sh\nansight test list --json\nansight test validate --json\n```\n\nRun only when requested. Add `--trace` when the result needs the full model context, tool payloads, and exportable execution graph rather than lightweight audit metadata:\n\n```sh\nansight test run --trace --json\nansight test run-all --trace --json\n```\n\nAgentic `test run`, `test run-all`, and the compatibility `test run-inline`\ncommand accept `--reasoning fast|balanced|deep`. Omitting it selects **Fast**, the\nnormal default for responsive execution. Balanced and Deep express progressively\ngreater emphasis on reasoning depth; actual model and provider budget come from\nthe server configuration. Every mode must satisfy the same requested steps and\nverification. Preserve the user's choice; do not automatically raise the mode\nafter a failure. For a requested Balanced run, for example:\n\n```sh\nansight test run --reasoning balanced --json\n```\n\nThis **Reasoning mode** is shared with app execution, replay, App Graph agent\nruns, and AI task extraction in Session Replay. The server resolves the model\nand provider reasoning effort from the client's configuration and selects the\ncredential handoff. No `--execution` flag is needed. Leave `--model-transport` at\nits default `auto` unless a transport override is requested for diagnostics.\nThe hidden legacy `--model` override is for diagnostics and cannot be combined\nwith explicit `--reasoning`. Reasoning is a run option, not a test-schema field.\n\nTest launches show simulator/emulator windows by default. When a local or CI\nrun should be windowless, add `--headless` to `test run` or `test run-all`; the\nflag applies to all selected targets and tool-requested device starts. It is a\nCLI option, not a test-schema field. `--json` and `--silent` do not imply it.\niOS skips opening Simulator.app; newly started Android emulators use\n`-no-window`. Existing windows, reused sessions, and physical devices are left\nalone.\n\nUse `ansight test inspect --json` for persisted results and `ansight test export --app-id ` when a detailed trace and its portable session archives must be retained or shared. Treat traces as potentially sensitive.\n\n## Tasks: Deterministic Reuse\n\nRead the current [Workspace Tasks](https://www.ansight.ai/docs/workspace/tasks) and [Task API](https://www.ansight.ai/docs/workspace/task-api-reference) before authoring. A task descriptor must remain statically extractable, its input schema must be rooted at an object, tool calls must be awaited serially, and at least one stable named `check` assertion must establish success. A normal return without assertions is `Inconclusive`.\n\n### Compose Existing Tasks\n\nBefore duplicating a stable workflow inside another task, inspect the workspace\ncatalog with `ansight.tasks.list({ query, feature, maxResults })`. Use the exact\ndiscovered `taskId`, and compose a task only when the candidate's complete\nbehavior, input schema, and postcondition match the dependency. Do not select a\ntask merely because its title or a fuzzy match looks plausible.\n\n```ts\ntype OpenGuideOutput = { guideVisible: boolean };\n\nconst catalog = await ansight.tasks.list({\n query: \"open area 3D guide\",\n feature: \"map\",\n maxResults: 10\n});\nconst child = catalog.tasks.find(candidate => candidate.taskId === \"map.open-guide\");\nif (!child) throw new Error(\"The guide task is unavailable.\");\n\nconst result = await ansight.tasks.run({\n taskId: child.taskId,\n input: { area: \"Secret Garden\" }\n});\nexpect(result.output?.guideVisible, { id: \"guide-opened\" }).toBe(true);\n```\n\n`ansight.tasks.list` and `ansight.tasks.run` need no `hostTools` declaration and\nmust be awaited serially like other task APIs. Both remain pinned to the\ncaller's exact repository, App ID, and live session; a child cannot select a\ndifferent repository, app, session, or device. Discovery uses the same task\nmetadata, input-schema, synonym, and conservative typo matching as external\ntask discovery. `maxResults` defaults to 10 and is capped at 20.\n\n`run` resolves only when the child status is `Passed`. Any other status rejects\nthe call and fails the parent unless the parent deliberately handles the error;\ncatch an error only when that failure is an explicitly supported branch. Each\nchild keeps its own run ID, assertions, timeout, action budget, tool-call audit,\nand persisted result. Each discovery or child execution costs one parent action,\nbut child actions consume only the child's budget. The parent's remaining\ndeadline caps the complete child call. Child assertions are not merged into the\nparent, so the parent must still make its own stable named assertion.\n\nThe host rejects direct self-calls and recursive call cycles and permits at most\neight task execution levels in one call chain. Use a shared TypeScript helper\ninstead of task composition when only pure in-process logic is being reused.\n\n### Ground UI Selectors In Recorded Evidence\n\nFor a task derived from a session, every selector must come from the UI evidence\ncaptured inside the selected extraction period. Build a selector inventory from\nthe visual-tree snapshots and, when text is visible but absent from those trees,\nretained screenshot OCR. Do not infer a selector from the task description or\nfrom implementation terminology.\n\nApply these rules to `nodeId`, `automationId`, `text`, `role`, `type`,\n`ancestorAutomationId`, and `action` fields used by `ansight.ui` or\n`ansight.keyboard` calls:\n\n- Every literal selector value must match a visible captured UI node. All fields\n in a compound selector must match the same observed node.\n- OCR can establish a `text` value only. It is not evidence for an automation\n ID, node ID, role, type, ancestor automation ID, or supported action.\n- Logs, telemetry, source code, framework navigation state, handoff text, task\n descriptions, and inferred component names are context, not selector\n evidence. A page, sheet, or view-model class name is not a `type` selector\n unless a captured UI node reports that exact type.\n- Prefer an observed automation ID. Otherwise use exact visible text, adding an\n observed role or ancestor only when needed to disambiguate it.\n- Keep evidence-derived selectors as inline string literals so they can be\n checked statically. Do not hide them behind variables, input values, spreads,\n computed keys, or helper-returned objects during extraction.\n- When no stable observed selector exists, leave a `REVIEW:` diagnostic or omit\n the unsupported step. Never fabricate a plausible selector or silently replace\n it with a coordinate tap.\n\nWhen the extraction surface offers **Require recorded selectors**, leave it\nenabled. Treat its selector-call scan as an additional validation pass: it must\nreject an unobserved literal or a dynamically assembled selector that cannot be\nproven against the selected period. If that validation is unavailable, enforce\nthe same invariant manually from `session trees` and retained screenshots. Do\nnot claim that an extracted task is grounded merely because it compiles.\n\nDiscover the exact path-derived task ID and run it only when its whole semantic behavior matches the request:\n\n```sh\nansight task list --app-id --repository --json\nansight task run --app-id --repository --session-id --json\n```\n\nTreat `Passed` plus its named assertions as completion evidence. Do not replay the same multi-step mutation manually after a terminal task result.\n\n### Execute Through An Existing Interactive Connection\n\nFor repeated live actions and task runs, keep one process and resident-host pipe\nopen instead of spawning a CLI per step:\n\n```sh\nansight app interact --session --repository --jsonl\n```\n\nRetain the process handle, read its automatic `ready.ui` observation, and send JSON\nLines on stdin:\n\n```json\n{\"id\":\"catalog\",\"command\":\"tasks\"}\n{\"id\":\"run\",\"command\":\"task\",\"taskId\":\"search.find\",\"input\":{\"query\":\"Kalymnos\"}}\n{\"id\":\"sequence\",\"command\":\"batch\",\"commands\":[{\"id\":\"query\",\"command\":\"type\",\"target\":{\"automationId\":\"search-field\"},\"value\":\"Kalymnos\"},{\"id\":\"submit\",\"command\":\"tap\",\"target\":{\"automationId\":\"search-submit\"}}]}\n```\n\nChoose IDs and inputs from the returned catalog; these are examples, not tasks\nguaranteed to exist. The repository, app and session stay fixed. Task execution\nuses the normal engine and returns its status, assertions and tool calls under\n`task`; only `Passed` succeeds. Evidence is automatic, including after a failed\ntask. No capture flag or separate screenshot request is needed.\nEach action also returns compact fresh `ui.nodes`; prefer semantic targets and\ntree-based verification, opening screenshots for visual checks or missing semantics.\nTargeted `type` focuses and replaces text by default (`replaceExisting:false`\nappends); untargeted `type` appends. Use observed IDs, not the example IDs above.\nDo not manually focus/type before a task that already owns those operations.\nInspect its source once if its full behavior is unclear from the catalog.\n\nBatches contain 1–32 known commands, execute serially, and stop on first failure.\nThey return ordered `results` and `skippedIds`, preserving evidence per action,\nplus top-level final-child `ui`, `screenshot`, and aggregate `timing`. Read child\nstatuses and final semantic state; inspect intermediate PNGs only when needed.\nIDs must be unique within the batch, nested batches and `exit` are forbidden,\nand each line is limited to 65536 characters. Do not batch flows that require\ndeciding targets from intermediate screenshots. A failed batch is not rolled\nback; never blindly replay completed or timed-out mutations. Automatic screenshots\nsettle within a bounded window; `ui_unsettled` retains evidence and stops the batch.\n`stable` or `unchanged` pixels are not proof of a task's semantic postcondition.\n\nTask timeout defaults to 120 seconds (`--task-timeout-ms`, maximum 300000); other\ncommands default to 15 seconds (`--timeout-ms`, maximum 60000). Timeout or\ndisconnect may interrupt capture and leave execution uncertain. `exit` or stdin\nEOF detaches without stopping the app/host. This direct bridge is not `app\nexecute`, an agentic test, or a new recording session. Use the live-operation\nskill for tree-first navigation with screenshot fallback, and keep execution authorization scoped\nto the requested task.\n\n`task run` and `repo task run` also accept `--headless` for device starts\nrequested by permitted host tools. They still require a connected session;\nthe flag does not authorize or perform an initial app launch for the task.\n\n### Extract A Task From Recorded Behavior\n\nAI task extraction in Session Replay offers **Reasoning mode: Fast / Balanced /\nDeep**, defaulting to Fast and using the same client configuration as agentic\nruns. Preserve the user's selection. The CLI `task extract` command below\ngenerates a deterministic draft from recorded events and does not use a model;\ndo not add `--reasoning` to it. Deterministic `task run` and `repo task run` also\nhave no reasoning mode.\n\nUse extraction when an existing session contains the exact deterministic behavior to preserve. Inspect its time bounds, touches, and visual-tree targets first:\n\n```sh\nansight session show --json\nansight session touches --json\nansight session trees --json\nansight task extract --start --end --workspace --title --json\n```\n\nKeep the extraction period narrow enough that the selector inventory represents\nthe behavior being preserved. Inspect representative beginning, middle, and end\nsnapshots when the period has many trees or screenshots. Treat the generated\nTypeScript as a draft. Review every `REVIEW:` comment, define the required\nstarting state, apply the recorded-evidence selector rules above, and replace the\ngenerated UI-stability check with a meaningful product outcome assertion.\nManually author unsupported long presses or multi-touch gestures; a tap without\na stable selector remains unresolved. Type-check, rediscover, and run the task\nagainst a connected development build before relying on it.\n\n### Debug A Failed Task Before Repairing It\n\nPreserve the failed task result and its evidence before editing or replaying.\nStart with the first failed tool call rather than the final exception or a nearby\napplication log:\n\n1. Map the failed tool name and occurrence number to the corresponding source\n call, and record its complete selector and state constraints.\n2. Use the tool-call timestamps to bracket the task run. Inspect UI evidence\n after the preceding successful action and through the failed call.\n3. Compare every visual-tree source available at those snapshots. Framework,\n native, and accessibility trees can expose different nodes. If a node exists\n only in the native tree, that proves the native node—not a missing framework\n class or inferred component type.\n4. Inspect retained screenshot OCR when visible text is missing from the trees.\n Keep the distinction between OCR text and structured selector fields.\n5. Search logs only to determine whether a failed selector came from\n implementation terminology. A string appearing only in logs is affirmative\n evidence that it was not grounded as a UI selector.\n6. Compare the UI before and after the preceding action and propose replacements\n only from nodes or OCR text that newly appeared or became relevant there.\n\nClassify the result before changing the task:\n\n| Evidence | Likely diagnosis | Repair direction |\n| --- | --- | --- |\n| Selector never appears in any tree or OCR result | Invented or stale selector | Replace it with an observed selector, or leave the step unresolved. |\n| Selector appears only in logs, source, or framework names | Implementation string used as UI evidence | Remove it; use captured UI text or a captured automation ID. |\n| Matching node exists, but not with the requested visibility, enabled state, role, ancestor, or action | State constraint or compound-selector mismatch | Correct only the constraint contradicted by the captured node. |\n| Matching node appears after the preceding action but after the wait timeout | Timing or transition problem | Wait on the observed postcondition and use a bounded timeout supported by replay evidence. |\n| Several nodes match | Ambiguous selector | Add an observed role, ancestor, or index only when the same evidence proves it. |\n| UI is visible in a screenshot or native tree but absent from a framework tree | Evidence-layer coverage gap | Use an exact selector exposed by an available tree, or OCR-backed visible text; do not invent the missing framework node. |\n\nIn Session Replay task extraction, run the draft against a connected session and\nuse **Debug why this task failed** when it is offered. Its failed-call mapping,\nselector-presence finding, and newly visible selector suggestions are evidence\nleads; verify the proposed selector against the retained tree or OCR provenance\nbefore accepting it. Outside that UI, use the `task run --json` result and the\nsession evidence commands above. Do not invent an `ansight task debug` command.\n\nAfter repairing the first supported cause, restore a known starting state and\nrun the task again. Stop after a terminal result; do not blindly repeat a timed\nout or partially completed mutation. A passing run must still contain the named\nproduct-outcome assertion, not merely successful tool calls.\n\n## Triggers: Event-Driven Evidence\n\nRead the current [Workspace Triggers](https://www.ansight.ai/docs/workspace/triggers) and [Trigger API](https://www.ansight.ai/docs/workspace/trigger-api-reference) before authoring. Keep the descriptor statically extractable, select one exact normalized event kind, and use declarative ANDed conditions. A handler may return no action or one bounded app action; it must not call unrelated host tools.\n\nInspect without executing code, then connect only with authorization:\n\n```sh\nansight repo automation inspect <app-id> <repository-root> --json\nansight repo automation connect <app-id> <repository-root> --json\nansight repo automation list --json\nansight repo automation runs <app-id> --json\nansight repo automation disconnect <app-id> --json\n```\n\nConnected triggers are trusted local code and persist with the linked workspace across resident-host restarts. External edits are not hot-reloaded; inspect again and reconnect after changing definitions. A missing run is not evidence of success.\n\n## Trends: Deterministic Metrics And Regression Evidence\n\nAn Ansight Trends definition starts when one matching SDK event arrives and completes when its matching end event arrives. Its inline `span` gives a meaningful app operation—such as launch, login, search, synchronization, or loading one item—a stable identity and time boundary for its metrics.\n\nInspect SDK event labels in source or captured session evidence before defining a span; never invent them. Treat the labels as a versioned instrumentation contract. Match optional event type or telemetry channel only when needed to disambiguate identical labels.\n\nA session can contain several completed instances of the same operation. Choose `firstCompleted`, `lastCompleted`, `exactlyOne`, or `all` deliberately for the intended analysis. When paired start and end events carry the same non-empty `details`, Ansight keeps that value as a group so repeated operations such as loading “Secret Garden” and “Seaside” form separate metric and regression series. Set `maximumDurationMs` high enough for a realistic slow run but low enough to prevent an unrelated later end event from closing an abandoned start.\n\nA Trends definition selects telemetry inside each chosen span, calculates deterministic `metrics`, and applies each metric's `budget`. Keep behavioral assertions in test validation and numeric trend rules in Trends definitions.\n\nAn optional `regression` belongs on the metric it monitors. Use [Trends](https://www.ansight.ai/docs/workspace/trends) for reference, regression, confirmation, and comparable-series rules. Inspect stored results with:\n\n```sh\nansight trends history --app-id <app-id> --json\n```\n\nChanging a regression policy does not rewrite prior decisions automatically. Preview with `ansight trends rebuild --app-id <app-id> --dry-run --json`; apply the rebuild only when the user asked to replace stored decisions under the current rules.\n\nValidation proves definitions and references are coherent. Only a finalized registered session produces metrics and historical comparison decisions.\n\n## Sanitizers: Privacy-Safe Exports\n\nRead [Sanitizers](https://www.ansight.ai/docs/workspace/sanitizers) before implementing a privacy policy. Use the generated typed sanitizer, return an edited item to retain it or `null` to remove it, fail closed for screenshots that neither OCR nor visual-tree text can inspect, and make an explicit keep-or-remove decision for binary artifacts.\n\nType-check the sanitizer, create a separate archive through `ansight session sanitize` or `ansight session export --sanitizer`, and inspect representative logs, network metadata, screenshots, trees, annotations, analyses, text artifacts, and binary artifacts in the result before sharing it. Never describe an uninspected sanitized archive as safe.\n\n## Validate The Complete Workspace\n\nRun only the checks whose files and configurations exist:\n\n```sh\nansight test validate <repository-root> --json\nnpx tsc -p ansight/tasks/tsconfig.json\nnpx tsc -p ansight/triggers/tsconfig.json\nnpx tsc -p ansight/sanitizers/tsconfig.json\n```\n\nResolve every validation warning. The runtime strips TypeScript types but does not type-check modules before execution. Re-run discovery after external edits, and reconnect triggers after changing them. Treat the installed host as authoritative when checked-in schemas or declarations disagree with it.\n\n## Report Results\n\nReport the workspace root, App ID and registration state, component type, path-derived ID, scaffold or files changed, validation and discovery performed, and whether execution was requested. For tests, report the outcome and whether `--trace` was enabled. For tasks, report terminal status and named assertions. For triggers, report connection and run state. For Trends, distinguish validation from an actual runtime evaluation. For sanitizers, report the output archive and inspection performed without claiming that the original session was modified.\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}