{"id":20048,"plugin_id":"plugins_6aa24265df788191a25e7db0cdeca894","kind":"skill","collection_source":null,"comparison_source":null,"observed_at":"2026-09-30T23:15:59.853Z","digest":"69e23eec5ea934250c0c8f67a533adc9ac617b6072531b243c105ba65a9f2133","against":null,"payload":{"name":"ansight-operate-live-app","description":"Operate an Ansight-connected app through its live lifecycle or visible semantic UI while preserving before-and-after evidence. Use to boot a target, launch or stop an app, resolve a live session, interact, or verify visible state; do not use merely to prepare the CLI, analyze a recording, or call app-internal remote tools.","included_files":[{"relative_path":"agents/openai.yaml","size_in_bytes":329}],"skill_md_contents":"---\nname: ansight-operate-live-app\ndescription: Operate an Ansight-connected app through its live lifecycle or visible semantic UI while preserving before-and-after evidence. Use to boot a target, launch or stop an app, resolve a live session, interact, or verify visible state; do not use merely to prepare the CLI, analyze a recording, or call app-internal remote tools.\n---\n\n## OpenAI plugin integration\n\nRun Ansight CLI commands using the execution tools in ChatGPT Work or Codex on the machine that owns the resident host. If this environment cannot execute commands or reach that host, explain the missing prerequisite and do not claim a live inspection succeeded. A remote workspace or cloud agent does not automatically have access to the developer’s local host. Resolve relative helper paths from this skill’s directory. When this workflow references another bundled skill, read its local SKILL.md completely before following it.\n\nUse these bundled files for the canonical skill URLs referenced below; keep public URLs when writing documentation for the user’s app:\n\n- https://www.ansight.ai/skills/ansight-install.md → [ansight-install](../ansight-install/SKILL.md)\n- https://www.ansight.ai/skills/ansight-cli-setup.md → [ansight-cli-setup](../ansight-cli-setup/SKILL.md)\n\n\n# Operate A Live App With Ansight\n\nEstablish one exact live session, observe before acting, perform the smallest authorized interaction, and verify the outcome from fresh evidence. Use `--json` for discovery and evidence commands, and keep the selected session ID explicit after resolution.\n\n## Prerequisites And Routing\n\nLive operation requires the Ansight SDK to be installed, initialized, and enabled in the app's development or QA build. Keep SDK enrollment, capture, and remote capabilities excluded from protected builds unless the app's documented policy explicitly permits them.\n\n- If the app does not contain a working Ansight SDK integration, follow `https://www.ansight.ai/skills/ansight-install.md` before attempting live operation.\n- If the `ansight` executable, resident host, or device dependencies are not ready, follow `https://www.ansight.ai/skills/ansight-cli-setup.md`.\n- If a development build is connected but the requested work needs app-internal tools, follow the Ansight Remote App Tools skill after resolving the session.\n\nDo not install or modify the SDK merely because no session is currently connected. First distinguish a missing integration from an app that is stopped, using the wrong build configuration, or temporarily disconnected.\n\n## Keep The Lifecycle Layers Separate\n\n| Layer | Start or attach | Stop | Meaning |\n| --- | --- | --- | --- |\n| Resident host | `ansight host status --json`; `ansight host run` | `ansight host stop` | Local capture and control service shared by sessions |\n| Device | `ansight device list --json`; `ansight device start ` | `ansight device shutdown ` | Simulator, emulator, or physical target |\n| App process | `ansight device launch ` | `ansight device terminate ` | Installed app instance |\n| Ansight session | The SDK connects when the enabled app reaches the host | `ansight session disconnect ` | Live evidence stream retained as a recording after disconnect |\n\nThere is no separate `ansight session start` command. Start the host and app, then discover the session created by the SDK connection. `session delete` removes retained data; it is not a stop command.\n\nDo not stop a host, device, or app that was already running unless the user asked. A resident host can be shared by other work.\n\n## Choose Device Window Behavior\n\nDevice launches show simulator/emulator windows by default. When the user wants\nwindowless operation, pass `--headless` on the command that starts the device:\n\n```sh\nansight device start ios --headless --json\nansight device start android --headless --json\n```\n\nThe same flag applies to automatic launches by `app execute`, `app-graph explore\n--launch`, replay, tests, and profiling, plus tool-requested starts during a\ncommand. iOS skips opening Simulator.app; new Android emulators use `-no-window`.\nIt does not close existing windows, restart running emulators, or affect physical\ndevices. Reusing a connected session leaves its window state unchanged.\n`--json`, `--silent`, and CI do not imply headless mode.\n\n## Establish The Exact Session\n\n1. Check the host:\n\n ```sh\n ansight host status --json\n ```\n\n If the CLI or workstation is not configured, follow the Ansight CLI Setup skill. If no host is running and live control is required, run `ansight host run` in a durable terminal and leave it attached.\n\n2. Inspect devices only when the app must be launched or the user named a target:\n\n ```sh\n ansight device list --all --json\n ```\n\n Start only the exact simulator or emulator needed. A physical iOS target is verified rather than booted.\n\n3. Launch the installed development or QA app when it is not already running:\n\n ```sh\n ansight device launch --json\n ```\n\n4. Discover and select the live session:\n\n ```sh\n ansight session list --connected --app-id --limit 20 --json\n ansight session show --json\n ```\n\n Match the App ID, device profile, platform, and connection state. If multiple sessions still match, do not choose the newest silently; use the user's supplied device or ask which instance is in scope.\n Follow `nextCursor` with `--cursor` only when the first filtered page does\n not contain the intended session, and keep the same filters on later pages.\n\n5. Record the session ID and initial observation timestamp or evidence IDs.\n\n## Fast Direct Exploration\n\nFor interactive exploration or human-style navigation, prefer one persistent process:\n\n```sh\nansight app interact --session --jsonl\n```\n\nRetain its process/PTY handle and write JSON Lines to stdin; do not start a new\nCLI process for every gesture. The resident-host pipe remains open. This is direct\ncontrol by the current agent, not `app execute` or a delegated agent.\n\nThe initial `ready` response and every action include a compact `ui` observation\nfrom fresh accessibility/visual trees, plus an automatically retained screenshot.\nPrefer `ui.nodes` to choose and verify semantic targets; no separate tree or\nscreenshot request is needed. Use exact automation IDs, or exact text narrowed by\nrole and `ancestorAutomationId`. The bridge resolves targets freshly and refuses\nmissing or ambiguous matches without input:\n\n```json\n{\"id\":\"query\",\"command\":\"type\",\"target\":{\"automationId\":\"search-field\"},\"value\":\"Kalymnos\"}\n{\"id\":\"select\",\"command\":\"tap\",\"target\":{\"text\":\"Kalymnos\",\"role\":\"button\"}}\n```\n\nThese IDs are examples; use those actually observed. Targeted `type` focuses and\nreplaces the field by default; `replaceExisting:false` appends. Untargeted `type`\nappends to the focused field. Never mix a target and coordinates. Trees are bounded\nto 128 visible semantic nodes and 256 characters per text field; `ui.truncated`\nmeans absence is not proof that a target does not exist. Use focused semantic\ninspection below for omitted details or longer exact selectors.\n\nView `screenshot.artifactPath` for custom-rendered controls, visual questions,\nmissing/insufficient tree semantics, or a mismatch between tree and expected state.\nThe PNG is an SDK app screenshot, not a generated image. Coordinates are a fallback\nnormalized to that screenshot (0–1):\n\n```json\n{\"id\":\"1\",\"command\":\"tap\",\"x\":0.5,\"y\":0.8}\n{\"id\":\"2\",\"command\":\"swipe\",\"x\":0.5,\"y\":0.8,\"endX\":0.5,\"endY\":0.2}\n{\"id\":\"3\",\"command\":\"pinch\",\"x\":0.5,\"y\":0.5,\"scale\":1.5}\n{\"id\":\"4\",\"command\":\"type\",\"value\":\"Kalymnos\"}\n{\"id\":\"5\",\"command\":\"back\"}\n```\n\nRead each result before deciding an unknown next target. `snapshot` optionally\nrefreshes asynchronous state; do not append it routinely after an action or task.\nEvidence capture is implicit. Screenshots use bounded visual settling (300 ms\nquietness, a 900 ms grace for unchanged screens, and a 2 s sampling window).\n`screenshot.settling.status` is `stable`, `unchanged`, or `timed_out`; quiet pixels\ndo not prove a network request or semantic goal completed. `ui_unsettled` stops a\nbatch and retains the latest screenshot; inspect before continuing, never replay\nthe action just to obtain evidence.\n`previousScreenshot` is the last observed frame, not a claim that nothing changed\nsince then. Reobserve when the app may have changed independently.\n\nFor a known sequence that needs no intermediate decisions, send one batch instead\nof paying a tool round trip for every action:\n\n```json\n{\"id\":\"search\",\"command\":\"batch\",\"commands\":[{\"id\":\"focus\",\"command\":\"tap\",\"x\":0.5,\"y\":0.8},{\"id\":\"query\",\"command\":\"type\",\"value\":\"Kalymnos\"}]}\n```\n\nBatches run in order, with automatic evidence for every action. The response has\nordered `results` and `skippedIds`; execution stops at the first failure, including\nmissing evidence or a timeout. Earlier actions are not rolled back. Never replay\nthe whole batch after partial completion. Use 1–32 commands with unique IDs\n(including the batch ID), no nested batches or `exit`, and at most 65536 characters\nper line. Check child statuses, then use the top-level `ui` and `screenshot` for\nthe final executed child's state. Inspect intermediate screenshots only when a\nfailure, visual question, or missing semantic evidence requires it. Top-level\n`timing` separates summed input/capture time from host batch wall time; it excludes\nthe agent's reasoning and tool round trips.\n\nTo reuse maintained tasks, pin the trusted repository when opening the process:\n\n```sh\nansight app interact --session --repository --jsonl\n```\n\n```json\n{\"id\":\"catalog\",\"command\":\"tasks\"}\n{\"id\":\"run\",\"command\":\"task\",\"taskId\":\"search.find\",\"input\":{\"query\":\"Kalymnos\"}}\n```\n\nUse the returned catalog's exact task ID and input schema. `task` uses the existing\ntask engine in the same app/session, captures resulting evidence automatically,\nand returns `task.status`, named assertions and tool calls. Only `Passed` is\nsuccessful; a normal return without assertions is `Inconclusive`. Task commands\ncan be batch children. Run only behavior the user authorized; opening the bridge\ndoes not authorize every available task. Default action timeout is 15 seconds;\ntask timeout is 120 seconds (`--timeout-ms` up to 60000 and `--task-timeout-ms` up\nto 300000). A timeout/disconnect can leave execution uncertain and fresh evidence\nunavailable; observe before deciding how to recover.\n\nBefore manually beginning a maintained multi-step flow, inspect the matching task's\ncomplete behavior once (including its source when the catalog is ambiguous).\nIf it enters a query and selects a result, invoke it with the query directly—do\nnot focus/type first and then have the task repeat those steps. If a user explicitly\nwants manual exploration, keep that route instead of running the same task afterward.\n\nFailures report whether input succeeded and whether evidence is available. Do not\nblindly repeat a timed-out action; it may already have executed. An `exit` command\nor stdin EOF detaches without stopping the app or host. Ctrl+C cancels the connection.\nIf the installed CLI/host does not support `app interact`, use the semantic workflow\nbelow and report the limitation; do not restart a shared host without authorization.\n\nUse focused semantic inspection below for assertions or details omitted from `ui`.\nDo not repeatedly request full trees or discover workspace tasks merely\nto navigate quickly when the user requested direct exploration.\n\n## Reuse Existing Workspace Behavior\n\nBefore manually performing a multi-step flow, make one focused repository-task search when a linked workspace exists and the request resembles maintained repeatable behavior:\n\nWhen already connected with `--repository`, send `tasks` on that connection and\nuse `task` to execute the selected behavior. Do not open a separate CLI process.\nOtherwise use:\n\n```sh\nansight task list --app-id --repository --json\n```\n\nRun a returned task only when its complete behavior and input schema exactly match the request. Treat a passed task and its named assertions as authoritative evidence; do not replay a failed, rejected, or inconclusive task manually and risk duplicating state changes. Skip task discovery for a simple one-step interaction or when no repository workspace is linked. Run a whole workspace test only when the user requested that scenario.\n\n## Observe Before Acting\n\nFor precise semantic workflows (as distinct from screenshot-driven exploration), start with:\n\n```sh\nansight ui snapshot --session --json\nansight ui find --session --automation-id --json\n```\n\nUse the narrowest stable selector available:\n\n1. exact `--automation-id`;\n2. exact text plus role;\n3. an ancestor-qualified selector;\n4. `--index` only after a fresh find proves the intended match.\n\nAll supplied selector fields constrain the same node. Treat node identifiers and result indexes as observation-scoped. Re-observe after navigation, dialogs, list changes, or other material state transitions.\n\nIf in-process framework or domain state would answer the question more directly, follow the Ansight Remote App Tools skill instead of guessing from the UI.\n\n## Interact And Capture Evidence\n\nUse one small action at a time:\n\n```sh\nansight ui tap --session --automation-id --json\nansight ui type --session --automation-id --value --json\nansight ui swipe --session --ancestor --direction up --length 0.6 --json\nansight ui pinch --session --automation-id --scale 1.8 --json\nansight ui back --session --json\n```\n\nUI actions use real device input and return structured before-and-after evidence. The persistent bridge is the tree-first fast path; these one-shot commands provide richer selectors and assertions. Raw `ansight input` is a fallback when these surfaces have a concrete capability gap.\n\nDo not mutate app state merely to make an assertion pass. A validation-only request authorizes observation, not repair.\n\n## Record Bounded Executions In Depth\n\n`app execute`, `app-graph run`, and `app-graph explore` accept\n`--reasoning fast|balanced|deep`. Omitting it selects **Fast**, the normal default\nfor responsive execution. Balanced and Deep express progressively greater\nemphasis on reasoning depth; actual model and provider budget come from the\nserver configuration. Every mode must satisfy the same requested steps and\nverification. Preserve the user's selected mode; do not automatically raise it\nafter a failure.\n\nThis is the same **Reasoning mode** used by agentic test runs, session replay, and\nAI task extraction in Session Replay. The server resolves the model and provider\nreasoning effort from the client's configuration and selects the credential\nhandoff. Leave `--model-transport` at its default `auto` unless a transport override\nis requested for diagnostics; no `--execution` flag is needed. The hidden legacy\n`--model` override is for diagnostics and cannot be combined with explicit\n`--reasoning`.\n\nFor example, when the user requests Deep:\n\n```sh\nansight app execute --reasoning deep --prompt \"\" --json\n```\n\nWhen the user needs a replayable or exportable account of an agent-driven run, add `--trace` to the command that performs it:\n\n```sh\nansight app execute --prompt \"\" --trace --json\nansight app execute --prompt \"\" --app-graph --trace --json\nansight app-graph run --trace --json\nansight app-graph explore --trace --json\n```\n\nFor `app execute`, `--trace` retains the full model context, assistant text, tool payloads, App Graph plan details, and exportable execution graph. App Graph `run` and `explore` use it to retain the full exportable agent trace. Without the flag, Ansight deliberately stores only lightweight audit metadata such as status, timing, token and call counts, and payload hashes.\n\nUse `--trace` when the run must be inspected in depth, replayed in the local player, exported, or used as detailed evaluation evidence. Treat the resulting trace as potentially sensitive because it can contain model context and tool payload bodies; do not enable it solely to obtain ordinary status or timing evidence.\n\n## Wait And Verify\n\nWait for a specific asynchronous state instead of sleeping:\n\n```sh\nansight ui wait --session --automation-id --condition visible --timeout-ms 15000 --json\n```\n\nThen verify the requested postcondition from fresh state:\n\n```sh\nansight ui assert --session --automation-id --expected-visible true --json\nansight ui snapshot --session --json\n```\n\nAn action response proves that input was attempted. It does not prove the requested outcome. Use an assertion, focused observation, or resulting session evidence for that conclusion.\n\n## End Only The Intended Layer\n\n- When the user wants an `app execute` run to close the app afterward, add `--close-app-on-completion`. It closes only that app process after success, failure, or cancellation, including when reusing an existing session. Multi-device runs close each selected app instance. The device, its window, and the resident host stay running. The default is to leave the app open; `--headless` controls windowless startup independently. A cleanup warning means the app may still be running; do not claim it closed without verification.\n- End the live evidence stream while retaining the recording: `ansight session disconnect --json`.\n- Stop the app process: `ansight device terminate --json`.\n- Shut down a virtual target only when the user requested it or this workflow exclusively started it.\n- Stop the resident host only when explicitly requested or when this workflow started a dedicated host and no other session depends on it.\n\nThe app may reconnect while its SDK remains active. After a requested stop, confirm with `ansight session list --connected --app-id --limit 20 --json`.\n\n## Report The Evidence Trail\n\nReport the selected App ID, device, and session; initial observation; each state-changing CLI command; fresh verification; important timestamps or evidence IDs; and any fallback outside Ansight. Distinguish observed state, executed action, and inferred explanation.\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}