{"id":47955,"plugin_id":"plugin_asdk_app_6abf60fccbc881919c2fea78a8056f5f","kind":"skill","collection_source":"plugin_package","comparison_source":null,"observed_at":"2026-10-24T13:35:04.642Z","digest":"8648876cdde9f2b640752db5f6837f65e56b046e077bfb3234b25cc0b54c42cf","against":null,"payload":{"description":"Define what \"good\" means for a process step and set the checks that hold it there — acceptance criteria, gates and monitors, target scores. Use for \"how do we know the AI is doing this right\", \"set up evals for this step\", \"what counts as a good output here\", or before handing any step to AI.","included_files":[],"name":"quality-bar","skill_md_contents":"---\nname: quality-bar\ndescription: Define what \"good\" means for a process step and set the checks that hold it there — acceptance criteria, gates and monitors, target scores. Use for \"how do we know the AI is doing this right\", \"set up evals for this step\", \"what counts as a good output here\", or before handing any step to AI.\n---\n\n# Quality bar\n\nFor a process owner who is about to let AI run part of a process, or who already has and\ncannot say whether it is working. This is where a business owner sets the bar in their own\nterms — *\"what counts as a good forecast category call\"*, *\"what a compliant response looks\nlike\"* — rather than delegating it to whoever builds the thing.\n\n## Read the model before you ask\n\nThe operating model already holds the owners, the steps, the timings, the constraints, the\nquality bars and the work in flight. **Ask the user only for what it doesn't have.**\nQuestioning someone about a process the model already describes wastes their time and\ntells them you didn't look.\n\nWhere the model is missing something, that absence is usually the finding — say what isn't\nrecorded rather than quietly filling the gap from conversation.\n\n## The rule this enforces\n\n**A step with no acceptance criteria cannot be handed to AI, because there is no bar to\nhold it to.** Not \"should not\" — cannot, meaningfully. You can wire it up, but you will\nhave no way to say whether it works, and no way to notice when it stops.\n\n`kowalah_get_process` reports `steps_missing_acceptance_criteria` and\n`steps_missing_output` in its `diagnosis` block. Those two counts are the backlog this\nskill works through.\n\n## Sequence\n\n1. **`kowalah_get_process`** — read the steps, their DEFINE contract\n   (`acceptance_criteria`, `input_artifact`, `output_artifact`), and any evals already\n   attached. Check `access`.\n2. **Fix the contract first** with `kowalah_save_process_step`. A step needs a defined\n   `output_artifact` before an acceptance criterion means anything — you cannot say what\n   good looks like until you have said what the thing *is*.\n3. **Then attach evals** with `kowalah_save_process_step_eval`.\n\nDoing 3 before 2 produces evals that check something nobody has defined.\n\n## Writing acceptance criteria that work\n\nThe test of a criterion: **could two people independently apply it and agree?**\n\n- *\"The response is accurate and helpful\"* — fails. Nothing to disagree with, nothing to\n  check.\n- *\"Every figure quoted traces to a row in the source ledger, and no figure appears that\n  isn't in it\"* — passes. Checkable by anyone, and it names the failure mode that actually\n  happens.\n\nWrite criteria against the **failure you are afraid of**, not the success you are hoping\nfor. Ask the owner: what would a bad output look like, and who would be harmed by it? That\nquestion produces better criteria than \"what does good look like\" in about nine cases out\nof ten.\n\n## Gate or monitor\n\n`mode` is the decision that matters most, and it is a business decision, not a technical\none.\n\n- **`gate`** — blocks the step until it passes. All-or-nothing. It **must not** carry a\n  `sampling_pct`; setting one is rejected, because a gate that only checks 20% of cases is\n  not a gate.\n- **`monitor`** — observes a sample and reports. May set `sampling_pct` (0–100).\n\nChoose a gate when a single bad output is unacceptable — a payment released, a regulated\ncommunication sent, a customer commitment made. Choose a monitor when you need to know the\nerror rate but individual errors are recoverable.\n\nMost steps want both: a narrow gate on the thing that must never go wrong, plus a monitor\non overall quality. They are separate eval rows on the same step.\n\n## The other eval fields\n\n- **`method`** — `human_review` (default) | `llm_judge` | `programmatic` | `golden_set`.\n  Start at `human_review` unless there is a reason not to; a human-reviewed eval that\n  actually runs beats an automated one nobody built. `golden_set` is the strongest option\n  where you have a set of known-correct cases to hold against.\n- **`criteria`** — free text, what \"pass\" means. This is where the owner's own words go.\n- **`target_score`** — the bar. **`current_score`** — the latest reading. Setting\n  `current_score` stamps the measurement time automatically; clearing it clears the stamp.\n  You cannot set the timestamp, the source, or the position — the server owns all three.\n  **Read that stamp back and check its age.** A `current_score` measured eleven months ago\n  is not a current score, and reporting it as one is how a step drifts for a year while the\n  dashboard says it's passing. Quote the reading with its date, or say plainly that nobody\n  has measured this since.\n- **`status`** — `defined` (default) | `active` | `retired`. An eval sitting at `defined`\n  is a written intention, not a running check. Be precise about the difference when\n  reporting: \"we have defined four evals and none are active\" is the honest version.\n- **`owner_user_id`** — the eval's owner on the client side. This matters: **once created,\n  the eval's owner may edit it thereafter even if the process itself sits above their\n  altitude.** So a function lead can own the quality bar on a process they do not otherwise\n  control. Set it deliberately.\n\n## Jidoka: autonomous steps need a stop-condition\n\nWhen a step's effective `target_disposition` is **`ai_autonomous`**, an\n`escalation_trigger` is **required** — the condition that returns control to a human. A\npartial update that would leave the step autonomous without a trigger is rejected.\n\nThis is not a form field to fill in. Ask the owner directly: *what has to happen for a\nperson to be pulled back in?* Good triggers are specific and observable — a confidence\nthreshold, a value above a limit, a case type, an eval failing. \"If something goes wrong\"\nis not a trigger.\n\nIf the owner cannot name one, that is a strong signal the step is not ready to be\nautonomous. Say so, and set the target to `ai_assisted` instead.\n\n## Permissions\n\nCreating an eval needs edit rights on the parent process. If `access.can_edit` is false,\nroute through `kowalah_propose_process_change` — a proposal to add a quality gate is\nexactly what that tool is for.\n\nOnce an eval exists, its `owner_user_id` may edit it regardless of process altitude. Admins\nand core team may do either anywhere in the organisation.\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}