Oximy Reality Checks

View saved version →

---
name: agent-autopsy
description: Reconstruct a completed, failed, expensive, or confusing agent session to find where its trajectory broke and what one change would prevent recurrence. Use when investigating post-hoc session diagnosis, tool-loop analysis, goal drift, premature completion, or repeated human rescue. Do not use as employee performance scoring.
license: MIT
---

# Agent Autopsy

Find the first consequential divergence, not merely the last visible error.

## Boundary

- Diagnose one bounded trajectory and preserve the source session unchanged.
- Do not score a person, infer intent, or convert tool counts into productivity.
- Do not implement proposed fixes unless the user separately asks for implementation.

## Workflow

1. Freeze the case. Record the session identifier, time range, original task, expected finish, and final observed state. Preserve the source transcript unchanged.
2. Run `scripts/session_stats.py` for a content-minimized timeline of tool calls, errors, repeated calls, and elapsed gaps. Treat statistics as leads, not diagnoses.
3. Reconstruct the trajectory as intent, decision, action, observation, and response. Include user corrections, permission denials, tool failures, context compaction, and handoffs.
4. Mark divergences: goal drift, assumption without evidence, wrong tool or target, ignored observation, retry without new information, instruction conflict, approval gap, premature completion, or verification at the wrong system boundary.
5. Identify the earliest point where a different decision would probably have changed the outcome. Distinguish triggering event, contributing conditions, and the final symptom.
6. Measure waste only from observable proxies: repeated tool calls, discarded artifacts, idle gaps, retries, review rounds, and reported cost. Do not convert tokens or elapsed time directly into productivity.
7. Propose at most three changes. Prefer one tighter task contract, one deterministic guard or verifier, and one instruction or permission change. Each proposal must cite the failure event it addresses.
8. Define a replay or fixture that would distinguish improvement from a cosmetically better explanation.

Read [references/failure-taxonomy.md](references/failure-taxonomy.md) when multiple failure classes overlap.

## Completion criterion

The original task and final state are explicit; the timeline accounts for every consequential branch; the first divergence is separated from the final symptom; each recommendation maps to evidence; and a repeatable check is defined for the highest-priority change.

## Output

1. One-paragraph finding
2. Case boundary and evidence coverage
3. Timeline of consequential events
4. First divergence, contributing conditions, and final symptom
5. Human interventions and recovery points
6. Observable waste
7. Up to three corrective changes
8. Replay plan and remaining unknowns

Referenced files: 4

View saved version →

---
name: bottleneck-shift
description: Compare a workflow before and after AI assistance to determine whether work disappeared or moved into review, waiting, correction, exception handling, or rework. Use when someone asks whether AI improved an end-to-end process rather than one production step. Do not claim causal impact from an uncontrolled before-and-after comparison.
license: MIT
---

# Bottleneck Shift

Measure the whole path to accepted completion. Faster creation can coexist with a slower system.

## Boundary

- Compare a named workflow and accepted completion event, not generic activity totals.
- Keep descriptive change separate from causal attribution and financial impact.
- Do not rank individual workers or infer personal productivity from phase timing.

## Required inputs

- A named workflow with a stable start and accepted completion event.
- Work-item or cohort evidence from a pre-AI baseline and an AI-assisted period.
- Phase timestamps or credible duration estimates.
- Quality and exception signals relevant to the work.

If the workflow definition, population, or completion event changed, record the mismatch before comparing periods.

## Workflow

1. Define the work item, start, accepted completion, phases, exceptions, and quality guardrails. Exclude activity that does not advance an item toward acceptance.
2. Assess comparability: volume, mix, staffing, seasonality, policy, tooling, and measurement changes. Label controlled, adjusted, directional, or not comparable.
3. Run `scripts/analyze_events.py` on a normalized event CSV when available. Inspect anomalies and missing spans before interpreting aggregates.
4. Compare creation, queue, review, correction, exception, escalation, rework, and total elapsed time. Include completion rate and quality guardrails.
5. Identify the constraint in each period. A phase with the most elapsed time is not automatically the constraint; test whether it limits accepted throughput.
6. Trace displaced work: which upstream effort fell, which downstream effort rose, and which role absorbed it.
7. Separate observation from explanation. Generate plausible mechanisms only after the measured changes are clear.
8. Recommend one operating change and its next measurement. Prefer a bounded experiment over a universal rollout conclusion.

Read [references/comparison-guide.md](references/comparison-guide.md) before comparing non-equivalent cohorts or reporting savings.

## Completion criterion

The unit of work and accepted completion event are fixed; both periods cover the same phase model or disclose differences; missing data and comparability limits are explicit; downstream review and rework are included; and any causal or savings language matches the study design.

## Output

1. Bottom line and evidence strength
2. Workflow and cohort definitions
3. Comparability assessment
4. Phase and throughput comparison
5. Quality, exception, and rework comparison
6. Work removed, work created, and role absorbing it
7. Candidate explanation, clearly labeled
8. Next operating change and measurement

Referenced files: 4

View saved version →

---
name: did-it-land
description: Reconcile an AI agent's completion claims with durable evidence in the relevant system of record. Use when someone asks whether agent work actually finished, shipped, sent, changed, or reached production. Do not use a passing local check as proof of an unobserved remote or real-world state.
license: MIT
---

# Did It Land?

Treat the agent's narration as claims, not evidence. Verify the post-condition in the system that owns the state.

## Boundary

- Verification is read-only by default. Do not create, resend, redeploy, approve, or otherwise mutate state to make a check pass.
- Do not perform a risky live test without explicit authorization.
- Keep local, committed, reviewed, merged, deployed and production-observed states distinct.

## Workflow

1. Recover the contract. Identify the original request, target, acceptance criteria, exclusions, and any later user corrections. Mark criteria that were never defined.
2. Extract completion claims from the agent's messages and artifacts. Split compound statements so each row can receive one verdict.
3. Name the system of record for each claim. Examples: filesystem, git commit, GitHub check, deployment provider, production URL, sent mailbox, calendar event, CRM record, payment ledger, or issue tracker.
4. Select an independent readback. Prefer a fresh API read, command, or direct artifact inspection over the same agent's summary. Never create or mutate state merely to verify it.
5. Capture evidence with timestamp, target identity, source, and the smallest excerpt or result needed. Inspect success envelopes: transport success is not application success.
6. Assign exactly one verdict from the evidence model below. A chain may contain different verdicts, such as committed = verified, deployed = verified, production behavior = unknown.
7. State the next smallest check needed for every partial, failed, or unknown claim. Do not quietly perform a risky live test.

Read [references/adapters.md](references/adapters.md) for common systems of record. Validate a JSON result with `scripts/validate_outcome.py`.

## Verdicts

- `verified`: fresh evidence directly establishes the claimed post-condition.
- `partial`: some, but not all, of the claimed state is established.
- `failed`: fresh evidence contradicts the claim.
- `claimed-only`: the agent stated it, but no independent evidence was collected.
- `unknown`: the required system, authority, or observation is unavailable.
- `not-applicable`: the recovered contract does not require this claim.

Do not convert `claimed-only` or `unknown` into failure. Do not convert a created artifact into proof that it was accepted, sent, merged, deployed, or used.

## Completion criterion

The check is complete when every acceptance criterion and every material completion claim appears in the evidence ledger; each has a named system of record, verdict, and evidence or explicit evidence gap; and the overall status is no stronger than its weakest required criterion.

## Output

Lead with one sentence: what is verified, what is not, and whether required work remains. Then provide:

1. Contract recovered
2. Evidence ledger
3. State chain, when work crosses multiple systems
4. Missing checks and why they were not run
5. Overall status: verified complete, partially verified, failed, or not verifiable

Referenced files: 4

View saved version →

---
name: human-review-tax
description: Compare an AI-produced artifact with the accepted final artifact and quantify the human correction and review it required. Use when someone asks whether AI saved work or shifted effort into editing, review, rework, or escalation. Do not infer time saved, authorship, or causality from a text diff alone.
license: MIT
---

# Human Review Tax

Measure the work between AI output and accepted output. A large diff is not automatically bad, and a small diff is not automatically correct.

## Boundary

- Analyze artifacts and workflow evidence, not individual reviewer performance.
- Do not infer effort, quality, authorship, savings or causality from diff size.
- Preserve both source artifacts; write derived reports or diffs to new files only.

## Required inputs

- The first AI-produced artifact or identifiable AI-assisted revision.
- The accepted final artifact, or the latest available revision clearly labeled as not yet accepted.
- The task and quality criteria used by the reviewer.

Optional evidence includes timestamps, comments, tracked changes, review rounds, ticket history, test failures, and the reviewer's own time estimate.

If the first AI artifact cannot be isolated, analyze the review process qualitatively and label attribution unknown.

## Workflow

1. Establish comparable versions. Record hashes, timestamps, formats, and acceptance status. Do not compare unrelated drafts merely because their filenames match.
2. Run `scripts/compare_artifacts.py` for a deterministic structural baseline. Its counts describe changed material, not effort or quality.
3. Review every material change and classify its primary reason: correctness, omission, unsupported claim, task misunderstanding, judgment, structure, style, policy or risk, formatting, or scope added after the AI draft.
4. Separate repair from preference. A reviewer choosing another valid style is not the same as fixing wrong work.
5. Reconstruct the review loop from available evidence: review rounds, elapsed time, active human time, blockers, reopens, and escalations. Keep reported time separate from inferred time.
6. Identify displaced work. Name the upstream effort AI reduced and the downstream work it created.
7. Recommend at most three changes tied to recurrent evidence: improve the task contract, constrain the AI step, add a verifier, change the handoff, or stop using AI for that part.

Read [references/classification.md](references/classification.md) before classifying changes in a high-stakes artifact.

## Completion criterion

Every material difference is classified or marked unresolved; repair is separated from preference and changed scope; time values identify their source; accepted status is explicit; and no statement of savings or causality exceeds the available evidence.

## Output

1. Net assessment in plain language
2. Artifact identity and comparison coverage
3. Review ledger by change category
4. Review loop: rounds, elapsed time, active time, and evidence source
5. Work removed versus work created
6. Repeated failure pattern, if the evidence supports one
7. Up to three workflow changes
8. Evidence limits

Referenced files: 4

View saved version →

---
name: policy-vs-reality
description: Compare written AI policy with observed agent configuration and runtime evidence, producing a non-accusatory mismatch ledger. Use when someone wants to test whether approved tools, data rules, permissions, retention, review gates, or prohibited actions match practice. Do not use individual activity as a proxy for intent or misconduct.
license: MIT
---

# Policy vs Reality

Test policy statements at the enforcement point they imply. A document can be current while implementation is missing; configuration can be compliant while runtime conformance remains unobserved.

## Boundary

- Default to system, workflow, or cohort evidence. Name individuals only when the user explicitly requires it and the evidence and policy authorize that use.
- Do not characterize a mismatch as misconduct. It may be policy ambiguity, stale documentation, configuration drift, an approved exception, or incomplete evidence.
- Do not change policy, configuration, permissions, or records during the comparison.

## Workflow

1. Establish scope: policy version, effective date, covered population, environments, workflows, and observation window.
2. Decompose the policy into atomic, testable statements. Separate requirements, prohibitions, approvals, exceptions, and aspirations.
3. For each statement, name its expected enforcement or evidence point: identity provider, model gateway, endpoint, device policy, agent configuration, tool permission, log, review system, retention control, or human procedure.
4. Inventory configuration and runtime evidence. Run `scripts/validate_matrix.py` after constructing the structured comparison matrix.
5. Assign one status per statement: aligned, configured-not-observed, observed-not-documented, contradictory, exception, ambiguous-policy, or unknown.
6. Distinguish collection, transmission, processing, storage, memory, display, reporting, and deletion. A control covering one does not silently cover the others.
7. Determine the likely mismatch class without assigning blame: policy defect, documentation lag, control gap, implementation drift, exception-management gap, or evidence gap.
8. Prioritize remediation by plausible harm, breadth, reversibility, and evidence strength. State the owner role and verification readback, not a person's name unless supplied.

Read [references/status-model.md](references/status-model.md) before finalizing the matrix.

## Completion criterion

Every in-scope policy statement is atomic and mapped to an evidence or enforcement point; every row has one status and an evidence pointer or explicit gap; lifecycle stages are not conflated; exceptions remain distinct from contradictions; and the output contains no unsupported allegation about individual behavior.

## Output

1. Scope and policy identity
2. Coverage and evidence limitations
3. Policy-to-reality matrix
4. Mismatches by class, not by person
5. Approved or unresolved exceptions
6. Prioritized remediation with owner role
7. Verification plan

Referenced files: 4

View saved version →

---
name: right-size-my-agent
description: Compare an agent's configured capabilities with observed tool use and propose a least-privilege configuration for defined workflows. Use when someone wants to reduce agent permissions, MCP access, filesystem scope, network reach, or approval bypasses. Do not apply permission changes or infer safe denial from non-use alone.
license: MIT
---

# Right-Size My Agent

Derive a proposed permission envelope from actual work, then challenge it with the workflows the agent must still perform.

## Boundary

- Produce a reviewable proposal; do not change permissions, credentials or agent configuration.
- Treat configured capability, observed use and required capability as separate states.
- Do not remove access solely because it is absent from a limited trace window.

## Required inputs

- The agent and workflows being scoped.
- Current configuration or a reliable capability inventory.
- A representative trace window, including exceptional work when available.

If traces are not representative, produce an exposure map and evidence plan rather than a least-privilege recommendation.

## Workflow

1. Define the unit: one agent identity, environment, user or service account, and named workflow set. Do not aggregate identities with different authority.
2. Inventory configured capability across tools, MCP servers, commands, filesystem roots, network destinations, credentials, approval policies, and downstream service roles.
3. Run `scripts/summarize_tool_use.py` on available JSONL traces. Verify high-risk and ambiguous calls in the raw evidence.
4. Map each observed capability use to the workflow step it enabled. Label capability not observed, observed, or required-but-not-observed.
5. Identify privilege gaps: write when read sufficed, broad path when one directory sufficed, reusable credential when one-action authorization sufficed, owner-level role when a narrower resource role sufficed, or external communication when a draft-only action sufficed.
6. Propose the narrowest envelope that supports the named workflows. Include explicit approval gates and a break-glass path where removal would otherwise make legitimate exceptional work impossible.
7. Run counterexamples against historical and anticipated tasks. State which tasks the proposal would block.
8. Produce a reviewable patch or configuration diff, but do not apply it without explicit authorization. After any later application, rerun representative tasks and inspect denials.

Read [references/capability-model.md](references/capability-model.md) before combining permissions across tools or identities.

## Completion criterion

Every configured capability is accounted for; every retained high-risk capability maps to a named workflow or break-glass case; non-use is not treated as proof of safety to remove; counterexamples are recorded; and all proposed mutations remain unapplied unless separately approved.

## Output

1. Scope and trace coverage
2. Configured capability map
3. Observed use by workflow
4. Excess or ambiguous privilege findings
5. Proposed envelope and approval gates
6. Tasks that may break
7. Reviewable patch, if the format is known
8. Validation plan and evidence limits

Referenced files: 4

View saved version →

---
name: safe-to-paste
description: Inspect text or files before they are shared with an AI service, make a task-preserving redacted copy, and explain residual exposure. Use when someone asks whether material is safe to paste or upload, or wants sensitive content sanitized for AI use. Do not treat the result as legal approval or a guarantee of vendor handling.
license: MIT
---

# Safe to Paste

Perform a local data-minimization review. The goal is not to make the source anonymous at any cost; it is to preserve the minimum information the requested AI task genuinely needs.

## Boundary

- Do not upload, paste, transmit, or quote the user's material into an external service as part of the review.
- Never modify the source file. Write a sibling copy only when the user asks for one.
- Treat scanner output as candidate detection. Names, trade secrets, privilege, and identifying combinations require semantic review.
- A vendor policy, enterprise contract, destination model, and retention setting can change the decision. If the destination is unknown, record it as unknown instead of inventing a policy.

## Workflow

1. Establish the intended AI destination and task. Record each as supplied, observed, or unknown. Do not block a local inspection when either is unknown.
2. Inspect the visible content and format-specific hidden surfaces. For office files, include metadata, comments, tracked changes, hidden sheets, speaker notes, and embedded filenames when the available document tools support them.
3. Build a sensitivity ledger. Account for credentials, direct identifiers, quasi-identifiers, customer or employer confidential material, regulated data, privileged material, and instructions that reveal protected business processes.
4. Decide what the task needs. For every sensitive element, choose keep, generalize, tokenize, remove, or escalate for human judgment. Consistent tokens such as `[CUSTOMER_1]` should preserve relationships without preserving identity.
5. If a redacted copy was requested, create it locally and run the deterministic scanner in `scripts/redact_text.py` against the copy. Do not claim that a clean scan proves anonymity.
6. Report residual risk and the evidence gaps that prevent a stronger conclusion. If the destination's handling terms matter, identify the exact term the user must verify rather than making a broad vendor claim.

Read [references/review-guide.md](references/review-guide.md) when classifying sensitive material or reviewing a non-plain-text file.

## Completion criterion

The review is complete only when the destination and task are recorded; every detected sensitive element has a disposition; the source remains unchanged; any redacted copy has been re-scanned; and residual risks and unknowns are explicit.

## Output

Return:

1. `Decision`: safe for the stated task, safe after listed changes, needs human approval, or insufficient evidence.
2. `Destination and task`: including unknowns.
3. `Sensitivity ledger`: item, location, category, disposition, reason.
4. `Redacted artifact`: path, if created.
5. `Residual risk`: what could still identify, expose, or mislead.
6. `Evidence limits`: what this review did not establish.

Referenced files: 4

View saved version →

---
name: skill-sunset
description: Decide whether an installed agent skill should be kept, narrowed, revised, merged, quarantined, or retired using real activation and outcome evidence. Use when skills are stale, conflicting, noisy, costly, unused, or suspected of worsening work. Do not delete or rewrite a skill without explicit approval.
license: MIT
---

# Skill Sunset

Evaluate the skill as a production dependency. Installation and popularity are not evidence that it improves this user's work.

## Boundary

- Inspect the exact installed version and preserve it during the review.
- Do not delete, disable, edit, publish or change marketplace state without separate authorization.
- Do not equate activation count, repository stars or marketplace installs with outcome quality.

## Workflow

1. Identify the exact skill version, source, installation scope, supported agents, and intended trigger. Hash the reviewed files.
2. Recover the skill's contract: what behavior it should change, when it should activate, when it should not, what artifact it should produce, and how completion is checked. Missing contract elements are findings, not assumptions.
3. Gather representative evidence: activation logs, sessions where it should and should not have fired, user corrections, outputs, review burden, tool calls, failures, and cost. Record coverage and selection bias.
4. Run `scripts/summarize_activations.py` when structured activation records exist. Inspect the source sessions for every high-impact result.
5. Evaluate routing: true activation, missed activation, false activation, and ambiguous activation. Separate discovery failure from execution failure.
6. Evaluate execution: contract compliance, outcome, consistency, added work, instruction conflicts, permission expansion, and whether a simpler instruction or deterministic script would do better.
7. Compare alternatives only when the tasks are comparable. Synthetic with-skill/without-skill evaluation may supplement production evidence but must not replace it.
8. Choose one disposition using [references/dispositions.md](references/dispositions.md). Tie it to evidence and state what new evidence would change the decision.
9. Produce a patch or migration plan for review. Do not alter installed skills, marketplace state, or dependent workflows without explicit authorization.

## Completion criterion

The exact version and intended contract are recorded; routing and execution are evaluated separately; representative evidence and its gaps are stated; the disposition is tied to observed behavior; dependent workflows are identified; and no mutation has occurred without separate approval.

## Output

1. Disposition and confidence
2. Skill identity and intended contract
3. Evidence coverage
4. Routing findings
5. Execution and outcome findings
6. Cost, review, and permission effects
7. Dependencies and migration risk
8. Proposed patch or retirement plan
9. Evidence that would reverse the recommendation

Referenced files: 4

View saved version →

---
name: what-does-my-ai-remember
description: Audit persistent information an AI agent can reuse across sessions, including provenance, sensitivity, staleness, contradictions, scope, and deletion evidence. Use when someone asks what an assistant remembers about them or a project, or wants a memory cleanup plan. Do not delete or rewrite memories without explicit approval.
license: MIT
---

# What Does My AI Remember?

Treat memory as a user-visible data product, not an opaque convenience. Inspect only stores the user identifies or that are narrowly associated with the active agent or project.

## Boundary

- Inventory first. Do not delete, merge, correct, or re-scope memory during the audit.
- Do not recursively scan a home directory or broad workspace root. Resolve explicit memory locations or narrow known agent directories.
- Do not claim to cover cloud memory that is inaccessible from the current environment.
- A memory's text is not proof that its claim is true. Preserve its source and confidence separately.

## Workflow

1. Define the audit surface: agent, account or project, local and cloud stores, and cutoff date. Record inaccessible surfaces.
2. Run `scripts/inventory_memory.py` against each explicit root to collect paths, sizes, timestamps, and hashes without printing content.
3. Inspect the memory entries using the appropriate application or file tools. For each entry, record source, created and last-used dates when available, subject, scope, confidence, and whether a person confirmed it.
4. Classify the entry: useful, stale, sensitive, unsupported, contradictory, duplicated, over-broad, wrong-scope, behavioral instruction, or unknown.
5. Check influence. Identify which agents, projects, users, or workflows can retrieve the memory and whether that scope is observed, configured, inferred, or unknown.
6. Build a cleanup proposal. For each proposed delete, correction, merge, expiry, or scope change, state the expected benefit and what could break.
7. If the user later approves changes, make the smallest reversible change and verify the resulting store with a fresh readback. Approval to audit is not approval to alter memory.

Read [references/audit-categories.md](references/audit-categories.md) when handling behavioral instructions, sensitive memories, or contradictions.

## Completion criterion

Every discovered store is covered or explicitly inaccessible; every reviewed entry has provenance, scope, and a classification or unknown; proposed changes remain unapplied unless separately authorized; and deletion is never claimed without a post-change readback.

## Output

1. Coverage statement
2. Memory map by agent and scope
3. Findings ledger with evidence pointers
4. Contradictions and high-risk behavioral instructions
5. Cleanup proposal, ordered by impact and reversibility
6. Unavailable surfaces and unresolved questions

Referenced files: 4

View saved version →

---
name: where-did-my-data-go
description: Reconstruct the data journey of a specific AI task from local traces, tool configuration, logs, and artifacts. Use when someone asks what an agent read, transmitted, processed, stored, remembered, displayed, or reported. Do not infer vendor retention or deletion from local traffic evidence alone.
license: MIT
---

# Where Did My Data Go?

Build a data lineage map for one bounded task or session. Keep observed movement separate from configured possibility and vendor-policy claims.

## Boundary

- Analyze the supplied or locally available evidence. Do not resend the sensitive content to test where it goes.
- Minimize excerpts. Prefer hashes, field names, record counts, endpoint hosts, and source-line references over reproducing content.
- Do not say `not stored` when the evidence establishes only no observed local write.
- A component being configured does not prove it was invoked. A tool call does not prove the remote system retained the payload.

## Workflow

1. Bound the journey by session, time window, task, user, and data item. Give the data item a neutral label.
2. Inventory evidence sources: transcript, tool calls, MCP configuration, environment, proxy logs, filesystem changes, memory writes, model/provider settings, and downstream artifacts.
3. Run `scripts/trace_inventory.py` on JSONL traces for a content-minimized event inventory. Inspect the source records for material events the generic parser cannot classify.
4. Build nodes for origin, local agent, model endpoint, MCP server, external service, local artifact, telemetry, cache, and memory. Add a hop only when evidence identifies both sides or mark an endpoint unknown.
5. Label each hop with exactly one lifecycle verb: `read`, `collect`, `transmit`, `process`, `cache`, `store`, `remember`, `display`, `report`, or `delete`.
6. Assign an evidence state: observed, configured, vendor-stated, inferred, contradicted, or unknown. Include timestamp and evidence pointer.
7. Test the lifecycle for gaps. Specifically ask what proves retention, deletion, training use, administrative access, geographic processing, logging, and downstream propagation. Usually several will remain unknown.
8. Produce the smallest risk-reduction actions tied to the observed route. Do not substitute a generic privacy checklist.

Read [references/evidence-model.md](references/evidence-model.md) before interpreting vendor statements or absence of events.

## Completion criterion

Every in-scope data source has an origin; every observed destination has an incoming hop; every hop has a lifecycle verb, evidence state, and pointer; configured paths are not presented as observed use; and retention, access, and deletion unknowns are explicit.

## Output

1. Plain-language journey summary
2. Mermaid or text flow map
3. Hop ledger
4. Storage and memory end states
5. Configured-but-unobserved paths
6. Unknowns and the evidence required to resolve them
7. Risk-reduction actions tied to specific hops

Referenced files: 4