skills/kora-workflow-builder/references/testing.md

# Testing Runtime Assets

Use the cheapest safe check that exercises the actual contract.

1. For pure transformation scripts, run the script locally with representative
   JSON input and verify `emitOutput` produces the expected JSON.
2. For script-backed service nodes, use
   `kora test node <workflow-name> <node-id> --workspace <dir> --input @<dir>/input.json --environment <environment> --json`
   with representative input, or run from the workspace directory and use
   `--workspace . --input @input.json --environment <environment>`. This includes
   service nodes whose operations call installed extension functions through
   `@kora/runtime-sdk`.
3. For a repeatable source-defined workflow-node test suite, add `kind: Test`
   YAML under `tests/` and run `kora test suite --workspace . --environment <environment> --json`.
   Use `kora test suite --release <release> --environment <environment> --json`
   to verify an immutable release. Suite tests are effect-free: service nodes
   that need secrets or extensions are skipped, human-assigned tasks fail as
   human targets, and agent tasks run only with managed model egress.

   `input` and `expected` each accept one of three sources: `inline`, a JSON
   `file` in release source, or a saved-run `ref`. Use
   `kora run export <run-id> --out <dir> --json` to materialise files. A
   ref has shape `ref: {run, node, attempt?}`. `node` is the workflow node ID;
   `attempt` defaults to `1` and selects later executions of a repeated node.
   Save the run first with
   `kora run save <run-id> --json`; an
   unsaved run or missing evidence makes the case invalid. `input.ref` reads
   the selected node attempt's first recorded input and `expected.ref` reads
   its final recorded output. Saving a run preserves its
   recorded evidence; it does not remove truncation or redaction already
   applied when the run was recorded.
   Remove a pin with `kora run unpin <run-id> --yes --json` when the
   evidence is no longer needed. Omit `--yes` for an interactive confirmation.
   Kora refuses removal while an immutable release test references the run and
   reports the blocking release IDs.
4. To run the full source-defined suite before deployment, configure the
   versioned project manifest in `kora.yaml`:

   ```yaml
   spec:
     tests:
       onDeploy: block       # block | warn | off
       minGatePassRate: 100
       environments:
         staging:
           onDeploy: warn
   ```

   `block` leaves the previous deployment live when the gate fails; `warn`
   records the same summary and proceeds. Do not bypass a blocking result with
   a deploy-time flag. Fix the source-defined tests or change the reviewed
   `spec.tests` policy in source and create a new release.
5. For artifact outputs, use a workflow-node test, a workflow-node test suite, or a real workflow run. Do
   not claim release creation, release validation, or deployment proves artifact
   capture.

A failed workflow-node test is a publish blocker. A failed workflow-node test suite
is also a publish blocker. Do not create a release, deploy, or tell the user the
workflow is ready unless the test passes or the user explicitly approves
publishing anyway after you warn it is likely to fail.
Source validation and node execution are separate checks. Passing source
validation does not prove the node executed. Treat execution-host, provider,
and configuration failures as retryable only when the structured test result
explicitly marks them retryable; never infer transience from the provider name,
an HTTP status, or the fact that the source is valid.
Classify failures before responding:

- source/validation error
- missing saved-run evidence
- missing extension/binding/grant
- missing connection/credential
- extension runtime error
- provider/API error

If `kora test node ... --json` reports missing runtime SDK extension support, call
that a Platform test-harness gap. Do not claim extension-backed service nodes are
untestable in the current authoring environment.

Do not claim readiness if a script was only typechecked but never executed with
representative input.